Nginx Sidecar for TLS/SSL Termination on Kubernetes

2023-12-22

2 minute read

If you’re not ready for a service mesh like Istio, Cillium or Nginx’s own service mesh, an easy way to implement end-to-end encryption from the application LB to your pod or TLS/SSL termination for your pods behind a network LB is a proxy sidecar using Nginx. I’m going to describe how to set this up using cert-manager to fetch a letsencrypt certificate and mount that to the Nginx base image.

Cert-manager will need to be setup before the next steps and that can be seen here.

Snippet from a deployment manifest for the nginx proxy container that is mounting the config and certificate keypair. Just need to make sure the secret created from the ingress manifest is the same name referenced in the volume.

deployment.yaml

 1...
 2containers:
 3    - name: nginx-proxy
 4      image: public.ecr.aws/nginx/nginx:1.23
 5      ports:
 6        - name: https
 7          containerPort: 443
 8      volumeMounts:
 9        - name: nginx-config
10          mountPath: /etc/nginx/conf.d
11          readOnly: true
12        - name: certs
13          mountPath: /certs
14          readOnly: true
15...
16volumes:
17    - configMap:
18        name: nginx
19      name: nginx-config
20    - name: certs
21      secret:
22        secretName: demo-app-cert

This example ingress manifest is using the AWS LB controller, but the parts to take note of are the TLS section and the cert-manager annotation that will tell cert-manager to generate a certificate and secret with the keypair that will be mounted in the Nginx sidecar.

ingress.yaml

 1apiVersion: networking.k8s.io/v1
 2kind: Ingress
 3metadata:
 4  name: demo-app
 5  namespace: sandbox
 6  labels:
 7    helm.sh/chart: demo-app-0.1.0
 8    app.kubernetes.io/version: "1.0.0"
 9    app.kubernetes.io/managed-by: Helm
10    app.kubernetes.io/name: demo-app
11  annotations:
12    cert-manager.io/cluster-issuer: letsencrypt-staging
13    alb.ingress.kubernetes.io/backend-protocol: HTTPS
14    alb.ingress.kubernetes.io/group.name: sandbox-app-external
15    alb.ingress.kubernetes.io/healthcheck-port: "443"
16    alb.ingress.kubernetes.io/healthcheck-protocol: HTTPS
17    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
18    alb.ingress.kubernetes.io/load-balancer-name: sandbox-app-external
19    alb.ingress.kubernetes.io/scheme: internet-facing
20    alb.ingress.kubernetes.io/target-type: ip
21spec:
22  ingressClassName: alb
23  rules:
24    - host: demo.sandbox.example.com
25      http:
26        paths:
27        - path: "/"
28          pathType: Prefix
29          backend:
30            service:
31              name: demo-app
32              port:
33                number: 443
34  tls:
35    - hosts:
36      - demo.sandbox.example.com
37      secretName: demo-app-cert

This configmap is our Nginx config file that will proxy traffic to the app listening on port 8000.

 1apiVersion: v1
 2kind: ConfigMap
 3metadata:
 4  name: nginx
 5data:
 6  image-app.conf: |
 7    server {
 8      listen              443 ssl http2;
 9      server_name         demo.sandbox.example.com;
10      ssl_certificate     /certs/tls.crt;
11      ssl_certificate_key /certs/tls.key;
12      ssl_session_cache   shared:SSL:10m;
13      ssl_session_timeout 1h;
14      ssl_buffer_size     8k;
15
16      location / {
17          proxy_pass         http://0.0.0.0:8000;
18          proxy_set_header   Host $host;
19          proxy_set_header   X-Real-IP $remote_addr;
20          proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
21          proxy_set_header   X-Forwarded-Host $server_name;
22          proxy_set_header   Upgrade $http_upgrade;
23          proxy_set_header   Connection 'upgrade';
24          proxy_cache_bypass $http_upgrade;
25      }
26    }