Configuring NGINX Ingress Controller in Bare Metal Kubernetes using MetalLB
In this guide, we'll walk through the process of setting up the NGINX Ingress Controller in a bare metal Kubernetes cluster using MetalLB. MetalLB provides a load-balancer implementation for environments that do not natively support one, such as bare metal clusters.
Prerequisites
Before you begin, ensure you have the following:
- A Kubernetes cluster up and running on bare metal nodes. kubectl configured to interact with your cluster. Helm (package manager for Kubernetes) installed.
Step 1: Install MetalLB MetalLB is a load balancer implementation for bare metal Kubernetes clusters. We'll start by installing MetalLB.
root@master:~# kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/namespace.yaml
root@master:~# kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/metallb.yaml
root@master:~# kubectl get pods -n metallb-system
NAME READY STATUS RESTARTS AGE
metallb-controller-665d96757f-kqlx4 1/1 Running 11 (23h ago) 26h
metallb-speaker-5z4cc 4/4 Running 18 (23h ago) 4d2h
metallb-speaker-qjrlt 4/4 Running 22 (23h ago) 4d2h
metallb-speaker-rmnln 4/4 Running 17 (24h ago) 4d2h
Create IPPool Address
root@master:~# cat metallb-system.yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: cheap
namespace: metallb-system
spec:
addresses:
- 172.16.16.150-172.16.16.200
Apply the above in metalb-system namespace
root@master:~# kubectl apply -f metallb-system.yaml
root@master:~# kubectl get IPAddressPool -n metallb-system
NAME AUTO ASSIGN AVOID BUGGY IPS ADDRESSES
cheap true false ["172.16.16.150-172.16.16.200"]
Step 2: Install NGINX Ingress Controller
1. Add the NGINX Helm repository:
root@master:/home/chicco# helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
"ingress-nginx" already exists with the same configuration, skipping
root@master:/home/chicco# helm search repo nginx
NAME CHART VERSION APP VERSION DESCRIPTION
ingress-nginx/ingress-nginx 4.10.1 1.10.1 Ingress controller for Kubernetes using NGINX a...
2. Install the NGINX Ingress Controller:
root@master:/home/chicco# kubectl create ns ingress-nginx
root@master:/home/chicco# helm install ingress-nginx ingress-nginx/ingress-nginx -n ingress-nginx
NAME: ingress-nginx
LAST DEPLOYED: Thu Jun 27 12:46:33 2024
NAMESPACE: ingress-nginx
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The ingress-nginx controller has been installed.
It may take a few minutes for the load balancer IP to be available.
You can watch the status by running 'kubectl get service --namespace ingress-nginx ingress-nginx-controller --output wide --watch'
An example Ingress that makes use of the controller:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example
namespace: foo
spec:
ingressClassName: nginx
rules:
- host: www.example.com
http:
paths:
- pathType: Prefix
backend:
service:
name: exampleService
port:
number: 80
path: /
# This section is only required if TLS is to be enabled for the Ingress
tls:
- hosts:
- www.example.com
secretName: example-tls
If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:
apiVersion: v1
kind: Secret
metadata:
name: example-tls
namespace: foo
data:
tls.crt: <base64 encoded cert>
tls.key: <base64 encoded key>
type: kubernetes.io/tls
3. Check the NGINX Ingress Controller ingress-nginx namespace
root@master:/home/chicco# kubectl get all -n ingress-nginx
NAME READY STATUS RESTARTS AGE
pod/ingress-nginx-controller-c8f499cfc-q25jq 1/1 Running 0 39s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/ingress-nginx-controller LoadBalancer 10.103.214.79 172.16.16.150 80:31311/TCP,443:32197/TCP 40s
service/ingress-nginx-controller-admission ClusterIP 10.97.68.133 <none> 443/TCP 40s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/ingress-nginx-controller 1/1 1 1 39s
NAME DESIRED CURRENT READY AGE
replicaset.apps/ingress-nginx-controller-c8f499cfc 1 1 1 39s
We can see a Loadbalancer IP is allocated to ingress controller via Metallb
root@master:/opt/kubernetes# kubectl create secret generic my-docker-secret --from-file=.dockerconfigjson=$HOME/.docker/config.json --type=kubernetes.io/dockerconfigjson -n ingress-test
secret/my-docker-secret created
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
run: blue-app
name: blue-app
spec:
replicas: 1
selector:
matchLabels:
run: blue-app
template:
metadata:
labels:
run: blue-app
spec:
containers:
- image: omvedi25/main-blue:v1
name: blue-app
imagePullSecrets:
- name: my-docker-secret
-----------------------------------
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
run: green-app
name: green-app
spec:
replicas: 1
selector:
matchLabels:
run: green-app
template:
metadata:
labels:
run: green-app
spec:
containers:
- image: omvedi25/main-green:v1
name: green-app
imagePullSecrets:
- name: my-docker-secret
-------------------------------------
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
run: red-app
name: red-app
spec:
replicas: 1
selector:
matchLabels:
run: red-app
template:
metadata:
labels:
run: red-app
spec:
containers:
- image: omvedi25/main-red:v1
name: red-app
imagePullSecrets:
- name: my-docker-secret
root@master:/opt/kubernetes# kubectl create ns ingress-test
root@master:/opt/kubernetes# kubectl apply -f nginx-deploy-red.yaml -n ingress-test
deployment.apps/red-app created
root@master:/opt/kubernetes# kubectl apply -f nginx-deploy-blue.yaml -n ingress-test
deployment.apps/blue-app created
root@master:/opt/kubernetes# kubectl apply -f nginx-deploy-green.yaml -n ingress-test
deployment.apps/green-app created
Export the Service
root@master:/opt/kubernetes/# kubectl expose deploy green-aap --port 80 -n ingress-test
root@master:/opt/kubernetes/# kubectl expose deploy red-aap --port 80 -n ingress-test
root@master:/opt/kubernetes/# kubectl expose deploy blue-aap --port 80 -n ingress-test
Verify The Service
root@master:/opt/kubernetes# kubectl get svc -n ingress-test
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
blue-app ClusterIP 10.109.208.98 <none> 80/TCP 25h
green-app ClusterIP 10.110.140.247 <none> 80/TCP 25h
red-app ClusterIP 10.98.179.2 <none> 80/TCP 25h
Deploy The Ingress Resource
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
name: ingress-resource
spec:
ingressClassName: nginx
rules:
- host: nginx.arobyte.tech
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: red-app
port:
number: 80
- path: /blue
pathType: Prefix
backend:
service:
name: blue-app
port:
number: 80
- path: /green
pathType: Prefix
backend:
service:
name: green-app
port:
number: 80
root@master:/opt/kubernetes# kubectl apply -f ingress-resource.yaml
root@master:~# kubectl get ingress -n ingress-test
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-resource nginx nginx.arobyte.tech 172.16.16.150 80 25h
Verify now its working as expected
root@master:~# curl http://nginx.arobyte.tech
<h1><font color=red>Welcome to Ingress Controller</font></h1>
root@master:~# curl http://nginx.arobyte.tech/blue
<h1><font color=blue>Welcome to Ingress Controller</font></h1>
root@master:~# curl http://nginx.arobyte.tech/green
<h1><font color=green>Welcome to Ingress Controller</font></h1>
Conclusion We have successfully configured the NGINX Ingress Controller in your bare metal Kubernetes cluster using MetalLB. This setup allows you to manage and expose your Kubernetes services with ease, providing a robust load balancing solution in a bare metal environment. Feel free to explore more advanced configurations and features of MetalLB and NGINX Ingress Controller to suit your specific needs. Happy Kubernetes-ing!