Kubernetes contains an internal DNS module that automatically discovers and assigns DNS names to individual containers when instructed. In practice, this works very well and there is room for customization.
However, when the time comes, we frequently need to expose some or all parts of the Kubernetes cluster to the public. For instance, if a cluster exists inside a public Cloud Provider like AWS or Google Cloud Platform, we would like to have a container service that interacts with this cloud provider and change any A Records to point to the nodes that expose those services.
This is what the ExternalDNS project does. ExternalDNS is a Kubernetes project with a main purpose of automatically creating DNS records for Ingress or Service resources.
In this tutorial we are following a step-by-step approach to configure ExternalDNS integration within the Free Tier of Platform9’s Managed Kubernetes backed up by a DigitalOcean Droplet.
Let’s get started.
Setting up the Platform9 Free Tier Cluster
Below are the brief instructions to get you up and running with a working Kubernetes Cluster from Platform9:
- Signup with Platform9
- Click the Create Cluster button and inspect the instructions. We need a server to host the Cluster.
- Create a few Droplets with at least 3gb Ram and 2vCPU’s. Follow instructions to install the pf9cli tool and prepping the nodes.
$ bash <(curl -sL http://pf9.io/get_cli) $ pf9ctl cluster prep-node -i
- Switch to the Platform9 UI and click the refresh button. You should see the new nodes in the list. Assign the first node as a master and the other ones as workers.
- Leave the default values in the next steps. Then create the cluster.
- Wait until the cluster becomes healthy. It will take at least 20 minutes to finish.
- Click on the API Access tab and select to download the kubeconfig button:
- Once downloaded export the config and test the cluster health:
$ export KUBECONFIG=/Users/itspare/Theo/Projects/platform9/example.yaml $ kubectl cluster-info
Kubernetes master is running at https://184.108.40.206 CoreDNS is running at https://220.127.116.11/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy Metrics-server is running at https://18.104.22.168/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy
Now you are ready to deploy the ExternalDNS service.
Domain Name and API Keys
In order to test the ExternalDNS, we need to assign a Domain name. I happen to have one spare for this tutorial.
As Digital Ocean does not act as a DNS registrar, you need to assign the nameservers of the domain registrar to point to the following entries:
ns1.digitalocean.com ns2.digitalocean.com Ns3.digitalocean.com
After that, go to the Networking tab of your DigitalOcean Dashboard and add the domain name there:
Do not assign any droplet there, as we will let the ExternalDNS handle that.
Next we need to create a Personal Access Token for the DigitalOcean API. Navigate to the API->Tokens and Keys and create a new API key. Note that token value, as it will be exposed only temporarily in the UI.
Setting Up ExternalDNS
Setting up the ExternalDNS is the easy part. We just need to define the manifest that consists of the following services:
- A Service Account for the ExternalDNS deployment.
- A Cluster Role with required RBAC permissions.
- A Cluster Role binding assigned to the previous Cluster Role and Service Account.
- The ExternalDNS pod deployment, passing as arguments the domain name and DigitalOcean Token value.
apiVersion: v1 kind: ServiceAccount metadata: name: external-dns
apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: external-dns rules: - apiGroups: [""] resources: ["services","endpoints","pods"] verbs: ["get","watch","list"] - apiGroups: ["extensions"] resources: ["ingresses"] verbs: ["get","watch","list"] - apiGroups: [""] resources: ["nodes"] verbs: ["list"]
apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: external-dns-viewer roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: external-dns subjects: - kind: ServiceAccount name: external-dns namespace: default
apiVersion: apps/v1 kind: Deployment metadata: name: external-dns spec: replicas: 1 selector: matchLabels: app: external-dns strategy: type: Recreate selector: matchLabels: app: external-dns template: metadata: labels: app: external-dns spec: serviceAccountName: external-dns containers: - name: external-dns image: registry.opensource.zalan.do/teapot/external-dns:latest args: - --source=service - --domain-filter=expressiveartsfair.com - --provider=digitalocean env: - name: DO_TOKEN value: "YOUR_DIGITALOCEAN_API_KEY"
You can either put the manifest all in the same file or in separate files. Then apply the configuration:
$ kubectl apply -f external-dns.yml serviceaccount/external-dns created clusterrole.rbac.authorization.k8s.io/external-dns created clusterrolebinding.rbac.authorization.k8s.io/external-dns-viewer created deployment.apps/external-dns created
Monitor the deployment status via the Platform9 UI or in the terminal:
When everything is healthy, we can deploy our first service to test the external DNS configuration.
Create a new file named nginx.demo.service.yml with the following contents:
apiVersion: apps/v1 kind: Deployment metadata: name: nginx spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - image: nginx name: nginx ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: nginx annotations: external-dns.alpha.kubernetes.io/hostname: demo.expressiveartsfair.com spec: selector: app: nginx type: NodePort ports: - protocol: TCP port: 80 nodePort: 30080
The important parts are:
- We used the external-dns.alpha.kubernetes.io/hostname: demo.expressiveartsfair.com annotation passing the custom domain we want to bind
- We used a NodePort type to expose the node IP address to the public.
Now anytime we assign the external DNS annotation in a service, the daemon will monitor that event and use the Digital Ocean API to update the DNS Records. In the following image you can see that the DNS entry was added to point to the Droplet that we exposed to the NodePort service:
Now you may notice one small thing. Currently we cannot use an external Load Balancer service, as the Platform9 Kubernetes distro is running inside a droplet. Thus, we cannot directly navigate to demo.expressiveartsfair.com as the NodePort is open in a different port (port 30080 as we specified in the configuration).
Nonetheless, if we navigate to demo.expressiveartsfair.com:30080 we can see the nginx welcome page as usual:
In order to make this work we have a few options:
- Create redirect rule
- We can use an Ingress controller like nginx-ingress-controller to create on-the-fly nginx configurations based on some host rules. This way we can point directly to the backend service host as defined in the definition.
In either case, we have the ExternalDNS service handling and updating all the corresponding DNS records without user intervention.
Follow the reverse step and destroy all created resources:
$ kubectl delete -f Nginx-service.yml $ kubectl delete -f external-dns.yml
Then invalidate the DigitalOcean Token you created earlier.