Træfik as Kubernetes Ingress Controller on ARM with automatic TLS certificates
TL;DR: How I configured Træfik with automatic TLS certificates from Let’s Encrypt as an Ingress Controller for my Kubernetes Cluster on a bare metal ARM hardware running in my living room.
Several blog posts have already described how they deployed Træfik as an Ingress Controller for Kubernetes with automatic TLS using Let’s Encrypt. While configuring it myself, I realized it wasn’t so straightforward for my setup. I decided to share with you how I made it work for me, and how you can customize this configuration to meet your needs.
In this blog post I’m not going to explain what an Ingress Controller is, for that please refer to the official documentation. You can also find a comprehensible and exhaustive description of Træfik at the project’s homepage.
An Kubernetes Cluster on ARM
I run a Kubernetes Cluster on a bare metal ARM hardware. My cluster consists of a mix of Orange Pi PCs and Raspberry Pi boards. In this cluster I carry out my experiments on deployment automation with CI/CD, monitoring, networking, storage, etc.
An early version of my Kubernetes cluster on ARM with cardboard.
I have a toy domain name homeservice.click pointing to my home router. I use the Dynu free dynamic DNS service which allows me to resolve my domain name to my home IP address. I don’t run any critical services in that domain, so it’s not a big deal in case my internet connection is down. I should probably be more worried about the possible security issues related to that setup. But I do my best and try to keep all my software up to date and not let any interesting ports be left unguarded. A relatively small price to pay for the enormous fun of having my own “cloud provider”. This setup was mostly guided by the Hypriot blog post “Setup Kubernetes on a Raspberry Pi Cluster easily the official way!” plus some additional tweaks.
Having this setup I wanted to guarantee that when I’m accessing the domain I’m accessing my own machines at home. So I decided to have TLS connections and use Let’s Encrypt to easily get a certificate for my domain. I also knew it would be fun figuring out how to configure it.
Ingress Controller and related manifests
I use Kubernetes 1.6 and RBAC authorization is enabled by default when setting up the cluster using kubeadm. I decided to stick with it to learn how it works. I defined a service account for the Træfik application and allowed this account to perform routing and proxying related operations in the Kubernetes API server. Here’s how my manifests for that look like:
I want the Træfik application to manage the Ingress resources across
all namespaces of my Kubernetes cluster. That’s why it’s a
with access to all my services, ingresses, and endpoints resources. I
believe there’s still too much power defined in this
I didn’t have the time yet to figure out the bare minimum
necessary. Since I’m not enabling any changes to be performed by the
Træfik service account, it shouldn’t be a big deal to leave it as it
I then define a
ConfigMap resource with the Træfik configuration. Here
we’ll need to add some information to enable the TLS certificate
You should replace email@example.com with your own
e-mail. Remember to also update the
main = example.com line. The
port 80 is also enabled here, so Træfik can redirect all the traffic
to the port 443. Another interesting field is the
"/etc/traefik/acme.json" line, this indicates where Træfik will save
the certificates generated by Let’s Encrypt. It should be a mount in
Pod. Where this mount will point is going to depend on your
setup. That’s my Træfik deployment manifest:
The deployment manifest has a few things you could consider
customizing for your own use. The first part to customize is the
nodeSelector field. Because of my setup I assigned the Traefik
to a specific node of my cluster. In my setup it’s required because I
need a fixed internal IP that I can configure in my router to redirect
all the traffic on the ports 80 and 443 to it. The
also complies with this requirement. Depending on where and how you
run your Kubernetes cluster you might not have this limitation.
Another interesting part of the deployment manifest to customize is
the “traefik-cache” volume. I use
hostPath type of volume because,
as I mentioned previously, I assigned the Traefik
Pod to a specific
node. This volume can also be an
emptyDir type, but every time your
Pod restarts, new certificates are going to be requested from Let’s
Encrypt. This might lead you to reach the rate limit in Let’s Encrypt
and stop you from getting certificates for your domain. The best
option is to either use
hostPath or a volume provisioned
specifically for this.
With little modification in those manifests you can also have TLS certificates assigned to your domain. Træfik will automatically manage everything necessary for it to work from there.
Exposing a service under a new subdomain
Traefik is now configured as our Ingress Controller and is watching the Kubernetes API for the creation of new Ingress resources. We can expose a service from the Kubernetes cluster to the internet with a Ingress resource manifest. First, let’s deploy an example application.
Our deployment have triggered the creation of 2
Pods of our example
application. Now we create a Service that will balance the traffic
Finally, we can expose the
example-app to the internet under a new
subdomain with an Ingress controller like:
After that, in a few seconds, it will be possible to access our
example-app through the URL
accessing the new subdomain for the first time, right after deploying,
the certificate will appear as invalid. The certificate for a new
subdomain takes a few minutes to be issued.
To clean up, you can run:
It might be silly, but since I have configured the HTTPS, I feel more confident to deploy applications to my Kubernetes Cluster. I also had fun figuring out how to make it work. I believe I don’t have a common setup, but it might be of help for you to know how it works and use it as reference for your own manifests.
A small improvement on my OrangePi PC boards organization.
In a future post I plan to dig into more details on how I have configured my Kubernetes Cluster and home network. Maybe it’ll serve as an inspiration for you to also have your own “cloud” at home to deploy your projects. If you’re interested in this, please let me know on Twitter (@rafaelcaricio) or in the comments section below. If you have any questions about anything I wrote here, I’ll be glad to answer.Post by: Rafael Caricio