Platform9 Blog

Integrating External DNS with Kubernetes

Kubernetes contains an internal DNS module that automatically discovers and assigns DNS names to individual containers when instructed. In practice, this works very well and there is room for customization.

However, when the time comes, we frequently need to expose some or all parts of the Kubernetes cluster to the public. For instance, if a cluster exists inside a public Cloud Provider like AWS or Google Cloud Platform, we would like to have a container service that interacts with this cloud provider and change any A Records to point to the nodes that expose those services.

This is what the ExternalDNS project does. ExternalDNS is a Kubernetes project with a main purpose of automatically creating DNS records for Ingress or Service resources.

In this tutorial we are following a step-by-step approach to configure ExternalDNS integration within the Free Tier of Platform9’s Managed Kubernetes backed up by a DigitalOcean Droplet.

Let’s get started.

Setting up the Platform9 Free Tier Cluster

Below are the brief instructions to get you up and running with a working Kubernetes Cluster from Platform9:

  1. Signup with Platform9
  2. Click the Create Cluster button and inspect the instructions. We need a server to host the Cluster.
  3. Create a few Droplets with at least 3gb Ram and 2vCPU’s. Follow instructions to install the pf9cli tool and prepping the nodes.
    $ bash <(curl -sL http://pf9.io/get_cli)
    $ pf9ctl cluster prep-node -i
  4. Switch to the Platform9 UI and click the refresh button. You should see the new nodes in the list. Assign the first node as a master and the other ones as workers.
  5. Leave the default values in the next steps. Then create the cluster.
  6. Wait until the cluster becomes healthy. It will take at least 20 minutes to finish.
  7. Click on the API Access tab and select to download the kubeconfig button:
  8. Once downloaded export the config and test the cluster health:
$ export KUBECONFIG=/Users/itspare/Theo/Projects/platform9/example.yaml
$ kubectl cluster-info
Kubernetes master is running at https://134.122.106.235
CoreDNS is running at https://134.122.106.235/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://134.122.106.235/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy

Now you are ready to deploy the ExternalDNS service.

Domain Name and API Keys

In order to test the ExternalDNS, we need to assign a Domain name. I happen to have one spare for this tutorial.

As Digital Ocean does not act as a DNS registrar, you need to assign the nameservers of the domain registrar to point to the following entries:

ns1.digitalocean.com
ns2.digitalocean.com
Ns3.digitalocean.com

After that, go to the Networking tab of your DigitalOcean Dashboard and add the domain name there:

Do not assign any droplet there, as we will let the ExternalDNS handle that.

Next we need to create a Personal Access Token for the DigitalOcean API. Navigate to the API->Tokens and Keys and create a new API key. Note that token value, as it will be exposed only temporarily in the UI.

Setting Up ExternalDNS

Setting up the ExternalDNS is the easy part. We just need to define the manifest that consists of the following services:

  1. A Service Account for the ExternalDNS deployment.
  2. apiVersion: v1
    kind: ServiceAccount
    metadata:
    name: external-dns
  3. A Cluster Role with required RBAC permissions.
  4. apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRole
    metadata:
    name: external-dns
    rules:
    - apiGroups: [""]
    resources: ["services","endpoints","pods"]
    verbs: ["get","watch","list"]
    - apiGroups: ["extensions"]
    resources: ["ingresses"]
    verbs: ["get","watch","list"]
    - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["list"]
  5. A Cluster Role binding assigned to the previous Cluster Role and Service Account.
  6. apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRoleBinding
    metadata:
    name: external-dns-viewer
    roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: external-dns
    subjects:
    - kind: ServiceAccount
    name: external-dns
    namespace: default
  7. The ExternalDNS pod deployment, passing as arguments the domain name and DigitalOcean Token value.
apiVersion: apps/v1
kind: Deployment
metadata:
name: external-dns
spec:
replicas: 1
selector:
matchLabels:
app: external-dns
strategy:
type: Recreate
selector:
matchLabels:
app: external-dns
template:
metadata:
labels:
app: external-dns
spec:
serviceAccountName: external-dns
containers:
- name: external-dns
image: registry.opensource.zalan.do/teapot/external-dns:latest
args:
- --source=service
- --domain-filter=expressiveartsfair.com
- --provider=digitalocean
env:
- name: DO_TOKEN
value: "YOUR_DIGITALOCEAN_API_KEY"

You can either put the manifest all in the same file or in separate files. Then apply the configuration:

$ kubectl apply -f external-dns.yml
serviceaccount/external-dns created
clusterrole.rbac.authorization.k8s.io/external-dns created
clusterrolebinding.rbac.authorization.k8s.io/external-dns-viewer created
deployment.apps/external-dns created

Monitor the deployment status via the Platform9 UI or in the terminal:

When everything is healthy, we can deploy our first service to test the external DNS configuration.

Create a new file named nginx.demo.service.yml with the following contents:

apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx
annotations:
external-dns.alpha.kubernetes.io/hostname: demo.expressiveartsfair.com
spec:
selector:
app: nginx
type: NodePort
ports:
- protocol: TCP
port: 80
nodePort: 30080

The important parts are:

  • We used the external-dns.alpha.kubernetes.io/hostname: demo.expressiveartsfair.com annotation passing the custom domain we want to bind
  • We used a NodePort type to expose the node IP address to the public.

Now anytime we assign the external DNS annotation in a service, the daemon will monitor that event and use the Digital Ocean API to update the DNS Records. In the following image you can see that the DNS entry was added to point to the Droplet that we exposed to the NodePort service:

Now you may notice one small thing. Currently we cannot use an external Load Balancer service, as the Platform9 Kubernetes distro is running inside a droplet. Thus, we cannot directly navigate to demo.expressiveartsfair.com as the NodePort is open in a different port (port 30080 as we specified in the configuration).

Nonetheless, if we navigate to demo.expressiveartsfair.com:30080 we can see the nginx welcome page as usual:

In order to make this work we have a few options:

  • Create redirect rule
  • We can use an Ingress controller like nginx-ingress-controller to create on-the-fly nginx configurations based on some host rules. This way we can point directly to the backend service host as defined in the definition.

In either case, we have the ExternalDNS service handling and updating all the corresponding DNS records without user intervention.

Cleaning up

Follow the reverse step and destroy all created resources:

$ kubectl delete -f Nginx-service.yml
$ kubectl delete -f external-dns.yml

Then invalidate the DigitalOcean Token you created earlier.

Avatar
Latest posts by Theo Despoudis (see all)

You may also enjoy

With Kubernetes, Every Company Can Achieve Agility and Scale of Cloud Without the Lock-in

By Chris Tozzi

Kubernetes Networking – An Implementer’s In-depth Guide [Webinar 2 Recap]

By Chris Tozzi

The browser you are using is outdated. For the best experience please download or update your browser to one of the following: