MetalLB Load Balancer for Bare Metal Kubernetes

What is MetalLB?

As per official documentation 

MetalLB is a load-balancer implementation for bare metal Kubernetes clusters, using standard routing protocols.

Now the question is, why do we need it?

In cases where you are deploying your Kubernetes Cluster outside Cloud solutions(AWS, GCP), you need a way to expose your service outside the cluster. You can use NodePort to expose your service using port range 30000–32767, but every time your customer needs to access your service, he/she needs to specify this higher-order port. If you are trying to use the LoadBalancer service in bare metal, you will see your service will be pending forever. In these scenarios, you can use MetlabLB.

How does it work?

If we take an example of a cloud environment, the creation and allocation of external IP is the cloud provider’s responsibility. In the case of a bare-metal environment, it’s the responsibility of MetalLB. MetalLB assigned the IP from the reserved pool of IP addresses, which we allocate via configmap(you will see this in the later configmap section). Once the external IP is assigned, it needs to redirect traffic from the external IP to the cluster, and for that, it uses a protocol like ARP or BGP.

Let see this with the help of an example

• Create a deployment

kubectl create deployment nginx --image=nginx
deployment.apps/nginx created

• Expose the deployment

kubectl expose deploy nginx --port 80 --type LoadBalancer
service/nginx exposed

• If you check the deployment the external-ip is in the pending state

kubectl get svc nginx          
NAME    TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
nginx   LoadBalancer   10.96.255.195   <pending>     80:30681/TCP   46s

To fix this issue, we can implement a solution like MetalLB. 

• First step is to apply the manifests, which will create the namespace metallb-system

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/namespace.yaml
namespace/metallb-system created

To verify it

kubectl get ns                                                                                    
NAME                 STATUS   AGE
default              Active   20h
kube-node-lease      Active   20h
kube-public          Active   20h
kube-system          Active   20h
local-path-storage   Active   20h
metallb-system       Active   4s

We will apply one more manifest in the next step, which will create the deployment, daemonset, role and cluster role binding, service account, and pod security for metallb in metallb-system namespace.

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/metallb.yaml
podsecuritypolicy.policy/controller created
podsecuritypolicy.policy/speaker created
serviceaccount/controller created
serviceaccount/speaker created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller created
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created
role.rbac.authorization.k8s.io/config-watcher created
role.rbac.authorization.k8s.io/pod-lister created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created
rolebinding.rbac.authorization.k8s.io/config-watcher created
rolebinding.rbac.authorization.k8s.io/pod-lister created
daemonset.apps/speaker created
deployment.apps/controller created

To verify it

kubectl get all -n metallb-system                                                               
NAME                              READY   STATUS                       RESTARTS   AGE
pod/controller-65db86ddc6-f7brp   1/1     Running                      0          2m13s
pod/speaker-htxfd                 0/1     CreateContainerConfigError   0          2m13s
pod/speaker-kcct8                 0/1     CreateContainerConfigError   0          2m13s
pod/speaker-m2svj                 0/1     CreateContainerConfigError   0          2m13s

NAME                     DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/speaker   3         3         0       3            0           kubernetes.io/os=linux   2m13s

NAME                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/controller   1/1     1            1           2m13s

NAME                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/controller-65db86ddc6   1         1         1       2m13s

• If you look at the pod, you have two types of the pod created after applying the manifests, controller, and speaker. The controller is the cluster-wide MetalLB controller and responsible for IP assignment. The speaker is a deamonset that is going to be installed on each node in your cluster and advertises services with assigned IPs using various advertising strategies.
• The first time you install the metallb, you will see this error(CreateContainerConfigError); to fix it, you need to create the secret. The memberlist secret contains the secretkey to encrypt the communication between speakers for the fast dead node detection.

kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
secret/memberlist created

• If you verify the pod again, it should be in running state now

kubectl get all -n metallb-system                                                                                
NAME                              READY   STATUS    RESTARTS   AGE
pod/controller-65db86ddc6-f7brp   1/1     Running   0          6m28s
pod/speaker-htxfd                 1/1     Running   0          6m28s
pod/speaker-kcct8                 1/1     Running   0          6m28s
pod/speaker-m2svj                 1/1     Running   0          6m28s

• As the last step, you need to define the configmap. You can set the operation mode to either Layer 2 or BGP and the external IP range depending upon your environment. For this demo, I will use the protocol as layer2 and addresses as 172.19.0.200–172.19.0.250 as my Kubernetes node is running in the subnet range(check the output of kubectl get nodes -o wide command).

kubectl get nodes -o wide        
NAME                            STATUS   ROLES    AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                                     KERNEL-VERSION     CONTAINER-RUNTIME
my-kind-cluster-control-plane   Ready    master   24h   v1.19.1   172.19.0.3    <none>        Ubuntu Groovy Gorilla (development branch)   5.4.0-65-generic   containerd://1.4.0
my-kind-cluster-worker          Ready    <none>   24h   v1.19.1   172.19.0.2    <none>        Ubuntu Groovy Gorilla (development branch)   5.4.0-65-generic   containerd://1.4.0
my-kind-cluster-worker2         Ready    <none>   24h   v1.19.1   172.19.0.4    <none>        Ubuntu Groovy Gorilla (development branch)   5.4.0-65-generic   containerd://1.4.0

• Configmap will look like this

cat cm.yaml                                  
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: my-ip-space
      protocol: layer2
      addresses:
      - 172.19.0.200-172.19.0.250

• Apply this configmap

kubectl apply -f cm.yaml                                                                        
configmap/config created

• Repeat the above step

kubectl create deployment nginx --image=nginx
deployment.apps/nginx created

kind kubectl expose deploy nginx --port 80 --type LoadBalancer
service/nginx exposed

• This time if you check the service, you will see that metallb assigned the external ip to it from the pool you defined under configmap.

kubectl get svc nginx                                    
NAME    TYPE           CLUSTER-IP     EXTERNAL-IP    PORT(S)        AGE
nginx   LoadBalancer   10.96.34.106   172.19.0.200   80:30403/TCP   20s

Wrapping Up

Metallb allows you to create Loadbalancer for your service without deploying your cluster in the cloud environment. So far, I have explored the layer2 mode, which doesn’t need any external hardware. Another mode is BGP, which is production-ready but requires changes on the network end. Please let me know if you have explored BGP mode or any other functionality provided by metallb, which I am missing in this blog.