Kubernetes

Install Ingress NGINX Controller on k3s

Manuel Stingl

• 3 min read
Install Ingress NGINX Controller on k3s
Photo by Lars Kienle on Unsplash

The ingress controller mainly allows you to perform HTTP traffic routing to your services depending on the host and/or path. With k3s ships Traefik as an ingress controller. Traefik absoluteley does a great job routing your HTTP traffic, but it isn't suitable for every usecase. As an alternative, there is NGINX, a well known HTTP server, which also has some nice additional capabilities.

Installing the Ingress NGINX Controller is pretty straigt forward, but there are some tweaks you have to apply, to perform a smooth transition from Traefik to NGINX, which you probably want to do in a production environment.

As helm is the most common and simplest way of deploying software in your k8s cluster, we chose this method.

Add the helm repository of Ingress NGINX in your CLI or, if you're using Rancher, in the UI:

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx

Before we continue with the installation, we should highlight the following aspects.

k3s' Traefik listens to every defined Ingress in the cluster. Ingress NGINX not. It's default is to only listen to Ingress, defined with a dedicated ingress class. You could now modify all your Ingress definitions, but this would cause Traefik to no longer listen to those. The better option is to declare in the helm values controller.watchIngressWithoutClass to true, so all Ingress definitions will get observed.

Next, we want to set our new Ingress NGINX as the default IngressClass. Should you use multiple IngressClasses, double check if this is what you want! It is configured in the helm values with controller.ingressClassResource.default set to true.

If you're having your kubernetes cluster behind an external ingress server / loadbalancer, you might want to add the following two cofigurations:

  • controller.config.enable-real-ip to true
    Enables the ngx_http_realip module. This causes NGINX to use the IP address in the X-Forwarded-For header as the clients IP.
  • controller.config.use-forwarded-headers to true
    Enables passing of all incoming X-Forwaded-* headers to your application. This is especially required if you're performing a https termination at your "first" ingress server and using http internally, so your site knows using the X-Forwarded-Proto header, that it is originally using http. (Make sure to set this header in your "first" ingress for the upstream.)

For all available configuration options, see https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/

If you want to apply all of those configurations, we end up with the following values:

controller:
  config:
    enable-real-ip: true
    use-forwarded-headers: true
  ingressClassResource:
    default: true
  watchIngressWithoutClass: true

Use those values to install your Ingress NGINX in the ingress-nginx namespace with the name ingress-nginx.

If you're using helm CLI, this can be done using the following command:

helm upgrade --install ingress-nginx ingress-nginx \
  --repo https://kubernetes.github.io/ingress-nginx \
  --namespace ingress-nginx --create-namespace

As soon as your pods are ready, you can point your traffic to the new nginx service. Ingress NGINX will create per default a LoadBalancer Service for you. Remember that you have to have some kind of LoadBalancer Service Manager in your kubernetes cluster for that feature. If not, take a look at MetalLB, which allows this kind of service using virtual IPs in your network.

If you're using services with an https upstream, you have to set a new annotation on your Ingress definitions: nginx.ingress.kubernetes.io/backend-protocol: HTTPS.
(For Traefik you might have used the annotation ingress.kubernetes.io/protocol: https.)

To finish the transition to Ingress NGINX, you want to disable the integrated Traefik from k3s. To achive this, update the k3s.service file on every server node in your k3s cluster. The default location is /etc/systemd/system/k3s.service. Add --disable traefik to the ExecStart command. The file will then look a bit like the following service config:

[Unit]
...

[Service]
...
ExecStart=/usr/local/bin/k3s server --disable traefik
...

Restart the k3s server to finalize your work. systemctl daemon-reload and systemctl restart k3s will do the job for you. Now you're ready to use your new Ingress NGINX, have fun!

Sources: https://kubernetes.github.io/ingress-nginx/deploy/

Sign up for more like this.

Enter your email
Subscribe