
MetalLB Load Balancer for Bare Metal Kubernetes
What is MetalLB?
As per official documentation
MetalLB is a load-balancer implementation for bare metal Kubernetes clusters, using standard routing protocols.
Now the question is, why do we need it?
In cases where you are deploying your Kubernetes Cluster outside Cloud solutions(AWS, GCP), you need a way to expose your service outside the cluster. You can use NodePort to expose your service using port range 30000–32767, but every time your customer needs to access your service, he/she needs to specify this higher-order port. If you are trying to use the LoadBalancer service in bare metal, you will see your service will be pending forever. In these scenarios, you can use MetlabLB.
How does it work?
If we take an example of a cloud environment, the creation and allocation of external IP is the cloud provider’s responsibility. In the case of a bare-metal environment, it’s the responsibility of MetalLB. MetalLB assigned the IP from the reserved pool of IP addresses, which we allocate via configmap(you will see this in the later configmap section). Once the external IP is assigned, it needs to redirect traffic from the external IP to the cluster, and for that, it uses a protocol like ARP or BGP.
Let see this with the help of an example
- Create a deployment
kubectl create deployment nginx --image=nginx
deployment.apps/nginx created
- Expose the deployment
kubectl expose deploy nginx --port 80 --type LoadBalancer
service/nginx exposed
- If you check the deployment the external-ip is in the pending state
kubectl get svc nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx LoadBalancer 10.96.255.195 <pending> 80:30681/TCP 46s
To fix this issue, we can implement a solution like MetalLB.
- First step is to apply the manifests, which will create the namespace metallb-system
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/namespace.yaml
namespace/metallb-system created
To verify it
kubectl get ns
NAME STATUS AGE
default Active 20h
kube-node-lease Active 20h
kube-public Active 20h
kube-system Active 20h
local-path-storage Active 20h
metallb-system Active 4s
We will apply one more manifest in the next step, which will create the deployment, daemonset, role and cluster role binding, service account, and pod security for metallb in metallb-system namespace.
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/metallb.yaml
podsecuritypolicy.policy/controller created
podsecuritypolicy.policy/speaker created
serviceaccount/controller created
serviceaccount/speaker created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller created
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created
role.rbac.authorization.k8s.io/config-watcher created
role.rbac.authorization.k8s.io/pod-lister created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created
rolebinding.rbac.authorization.k8s.io/config-watcher created
rolebinding.rbac.authorization.k8s.io/pod-lister created
daemonset.apps/speaker created
deployment.apps/controller created
To verify it
kubectl get all -n metallb-system
NAME READY STATUS RESTARTS AGE
pod/controller-65db86ddc6-f7brp 1/1 Running 0 2m13s
pod/speaker-htxfd 0/1 CreateContainerConfigError 0 2m13s
pod/speaker-kcct8 0/1 CreateContainerConfigError 0 2m13s
pod/speaker-m2svj 0/1 CreateContainerConfigError 0 2m13s
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/speaker 3 3 0 3 0 kubernetes.io/os=linux 2m13s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/controller 1/1 1 1 2m13s
NAME DESIRED CURRENT READY AGE replicaset.apps/controller-65db86ddc6 1 1 1 2m13s
- If you look at the pod, you have two types of the pod created after applying the manifests, controller, and speaker. The controller is the cluster-wide MetalLB controller and responsible for IP assignment. The speaker is a deamonset that is going to be installed on each node in your cluster and advertises services with assigned IPs using various advertising strategies.
- The first time you install the metallb, you will see this error(CreateContainerConfigError); to fix it, you need to create the secret. The memberlist secret contains the secretkey to encrypt the communication between speakers for the fast dead node detection.
kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
secret/memberlist created
- If you verify the pod again, it should be in running state now
kubectl get all -n metallb-system
NAME READY STATUS RESTARTS AGE
pod/controller-65db86ddc6-f7brp 1/1 Running 0 6m28s
pod/speaker-htxfd 1/1 Running 0 6m28s
pod/speaker-kcct8 1/1 Running 0 6m28s
pod/speaker-m2svj 1/1 Running 0 6m28s
- As the last step, you need to define the configmap. You can set the operation mode to either Layer 2 or BGP and the external IP range depending upon your environment. For this demo, I will use the protocol as layer2 and addresses as 172.19.0.200–172.19.0.250 as my Kubernetes node is running in the subnet range(check the output of kubectl get nodes -o wide command).
kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
my-kind-cluster-control-plane Ready master 24h v1.19.1 172.19.0.3 <none> Ubuntu Groovy Gorilla (development branch) 5.4.0-65-generic containerd://1.4.0
my-kind-cluster-worker Ready <none> 24h v1.19.1 172.19.0.2 <none> Ubuntu Groovy Gorilla (development branch) 5.4.0-65-generic containerd://1.4.0
my-kind-cluster-worker2 Ready <none> 24h v1.19.1 172.19.0.4 <none> Ubuntu Groovy Gorilla (development branch) 5.4.0-65-generic containerd://1.4.0
- Configmap will look like this
cat cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
address-pools:
- name: my-ip-space
protocol: layer2
addresses:
- 172.19.0.200-172.19.0.250
- Apply this configmap
kubectl apply -f cm.yaml
configmap/config created
- Repeat the above step
kubectl create deployment nginx --image=nginx
deployment.apps/nginx created
kind kubectl expose deploy nginx --port 80 --type LoadBalancer service/nginx exposed
- This time if you check the service, you will see that metallb assigned the external ip to it from the pool you defined under configmap.
kubectl get svc nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx LoadBalancer 10.96.34.106 172.19.0.200 80:30403/TCP 20s
Wrapping Up
Metallb allows you to create Loadbalancer for your service without deploying your cluster in the cloud environment. So far, I have explored the layer2 mode, which doesn’t need any external hardware. Another mode is BGP, which is production-ready but requires changes on the network end. Please let me know if you have explored BGP mode or any other functionality provided by metallb, which I am missing in this blog.
Tag:aws, cloud, devops, kubernetes, loadblancer, metlallb