Integrating External DNS with Kubernetes

Kubernetes contains an internal DNS module that automatically discovers and assigns DNS names to individual containers when instructed. In practice, this works very well and there is room for customization.

However, when the time comes, we frequently need to expose some or all parts of the Kubernetes cluster to the public. For instance, if a cluster exists inside a public Cloud Provider like AWS or Google Cloud Platform, we would like to have a container service that interacts with this cloud provider and change any A Records to point to the nodes that expose those services.

This is what the ExternalDNS project does. ExternalDNS is a Kubernetes project with a main purpose of automatically creating DNS records for Ingress or Service resources.

In this tutorial we are following a step-by-step approach to configure ExternalDNS integration within a Platform9’s Managed Kubernetes cluster backed up by a DigitalOcean Droplet.

Let’s get started.

Pre-requisites

Kubernetes Cluster
Permissions to create resources

Domain Name and API Keys

In order to test the ExternalDNS, we need to assign a Domain name. I happen to have one spare for this tutorial.

As Digital Ocean does not act as a DNS registrar, you need to assign the nameservers of the domain registrar to point to the following entries:

ns1.digitalocean.com
ns2.digitalocean.com
Ns3.digitalocean.com

After that, go to the Networking tab of your DigitalOcean Dashboard and add the domain name there:

Do not assign any droplet there, as we will let the ExternalDNS handle that.

Next we need to create a Personal Access Token for the DigitalOcean API. Navigate to the API->Tokens and Keys and create a new API key. Note that token value, as it will be exposed only temporarily in the UI.

Setting Up ExternalDNS

Setting up the ExternalDNS is the easy part. We just need to define the manifest that consists of the following services:

1. A Service Account for the ExternalDNS deployment.

apiVersion: v1
kind: ServiceAccount
metadata:
name: external-dns

1. A Cluster Role with required RBAC permissions.

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: external-dns
rules:
- apiGroups: [""]
resources: ["services","endpoints","pods"]
verbs: ["get","watch","list"]
- apiGroups: ["extensions"]
resources: ["ingresses"]
verbs: ["get","watch","list"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["list"]

1. A Cluster Role binding assigned to the previous Cluster Role and Service Account.

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: external-dns-viewer
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: external-dns
subjects:
- kind: ServiceAccount
name: external-dns
namespace: default

The ExternalDNS pod deployment, passing as arguments the domain name and DigitalOcean Token value.

apiVersion: apps/v1
kind: Deployment
metadata:
name: external-dns
spec:
replicas: 1
selector:
matchLabels:
app: external-dns
strategy:
type: Recreate
selector:
matchLabels:
app: external-dns
template:
metadata:
labels:
app: external-dns
spec:
serviceAccountName: external-dns
containers:
- name: external-dns
image: registry.opensource.zalan.do/teapot/external-dns:latest
args:
- --source=service
- --domain-filter=expressiveartsfair.com
- --provider=digitalocean
env:
- name: DO_TOKEN
value: "YOUR_DIGITALOCEAN_API_KEY"

You can either put the manifest all in the same file or in separate files. Then apply the configuration:

$ kubectl apply -f external-dns.yml
serviceaccount/external-dns created
clusterrole.rbac.authorization.k8s.io/external-dns created
clusterrolebinding.rbac.authorization.k8s.io/external-dns-viewer created
deployment.apps/external-dns created

Monitor the deployment status.

When everything is healthy, we can deploy our first service to test the external DNS configuration.

Create a new file named nginx.demo.service.yml with the following contents:

apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx
annotations:
external-dns.alpha.kubernetes.io/hostname: demo.expressiveartsfair.com
spec:
selector:
app: nginx
type: NodePort
ports:
- protocol: TCP
port: 80
nodePort: 30080

The important parts are:

We used the external-dns.alpha.kubernetes.io/hostname: demo.expressiveartsfair.com annotation passing the custom domain we want to bind
We used a NodePort type to expose the node IP address to the public.

Now anytime we assign the external DNS annotation in a service, the daemon will monitor that event and use the Digital Ocean API to update the DNS Records. In the following image you can see that the DNS entry was added to point to the Droplet that we exposed to the NodePort service:

Now you may notice one small thing. Currently we cannot use an external Load Balancer service, as the Platform9 Kubernetes distro is running inside a droplet. Thus, we cannot directly navigate to demo.expressiveartsfair.com as the NodePort is open in a different port (port 30080 as we specified in the configuration).

Nonetheless, if we navigate to demo.expressiveartsfair.com:30080 we can see the nginx welcome page as usual:

In order to make this work we have a few options:

Create redirect rule
We can use an Ingress controller like nginx-ingress-controller to create on-the-fly nginx configurations based on some host rules. This way we can point directly to the backend service host as defined in the definition.

In either case, we have the ExternalDNS service handling and updating all the corresponding DNS records without user intervention.

Cleaning up

Follow the reverse step and destroy all created resources:

$ kubectl delete -f Nginx-service.yml
$ kubectl delete -f external-dns.yml

Then invalidate the DigitalOcean Token you created earlier.

Author
Recent Posts

Platform9

Platform9 is the only container management and orchestration solution that supports the complete lifecycle of the cloud-native journey on infrastructure anywhere. Our unique management plane simplifies operations, including observability & monitoring, increases uptime, minimizes costs, and scales as you scale. Built on open foundations, our platform enables enterprises to keep up with the ever-changing cloud-native ecosystem. Our cloud-native wizards provide Always-on Assurance™ every step of the way to ensure a smooth, simplified journey. Innovative enterprises like Juniper, Kingfisher Plc, Mavenir, Redfin, and Cloudera achieve 4x faster time-to-market, up to 90% reduction in operational costs, and 99.99% uptime.Platform9 is a better way to go cloud native, paving the way for an open distributed cloud