Kubernetes HA Cluster Setup using kubeadm (EC2 + HAProxy)
This project demonstrates how to set up a High Availabile Kubernetes Cluster using:-
kubeadm
3 Control Plane Nodes(t2.medium Ubuntu)
HAProxy Load Balancer (t3.micro Ubuntu)
Worker Nodes (t3.micro Ubuntu)
Spin 3 t2.medium EC2 instances IN AWS for Kubernetes Control Plane Nodes:-
==========================================================
Run below commands on all servers
---------------------------------------------
sudo apt update && sudo apt upgrade -y
Disable swap
--------------------
sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab
Enable kernel modules
===================================
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
sudo modprobe overlay
sudo modprobe br_netfilter
Sysctl settings
===================================
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF
sudo sysctl --system
Install Container Runtime - ContainerD
===================================
apt install -y containerd
mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml
Enable Systemd cgroup
====================================
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
sudo systemctl restart containerd
sudo systemctl enable containerd
Install Kubernetes Components
====================================
sudo apt install -y apt-transport-https ca-certificates curl
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo tee /etc/apt/keyrings/kubernetes-apt-keyring.asc
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.asc] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /" | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo chmod 644 /etc/apt/sources.list.d/kubernetes.list
sudo apt update
Install Kubelet, Kubeadm, kubectl
===============================================
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
Spin Up a t3.micro server in AWS for HAPROXY
HAPROXY Server Setup (Ip Address: 172.31.46.120)
=================
sudo apt install -y haproxy
---Edit config:--------
sudo nano /etc/haproxy/haproxy.cfg
-------Add at bottom:----------
frontend kubernetes bind *:6443 mode tcp option tcplog default_backend k8s-masters
backend k8s-masters mode tcp balance roundrobin option tcp-check server m1 172.31.36.186:6443 check server m2 172.31.42.16:6443 check server m3 172.31.33.101:6443 check
# ==================== Kubernetes API Server Frontend ====================
frontend k8s-api-frontend
bind *:6443
mode tcp
option tcplog
default_backend k8s-masters
# ==================== Kubernetes Masters Backend ====================
backend k8s-masters
mode tcp
balance roundrobin
# Health check - important!
option tcp-check
tcp-check connect port 6443
# Server definitions
server m1 172.31.36.186:6443 check fall 3 rise 2
server m2 172.31.42.16:6443 check fall 3 rise 2
server m3 172.31.33.101:6443 check fall 3 rise 2
systemctl restart haproxy
=============================================================
Clean up iptables (important on AWS)
sudo iptables -F
sudo iptables -t nat -F
sudo iptables -t mangle -F
sudo iptables -X
Initialize First Control Plane Node
==============================================================
sudo kubeadm init \
--control-plane-endpoint "172.31.46.120:6443" \
--upload-certs \
--pod-network-cidr=192.168.0.0/16 \
--node-name $(hostname -s)
Setup kubeconfig
=============================================
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Install CNI (Calico)
=============================================
kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
Command to regenerate the token if you forget to copy the token:-
=============================================
kubeadm token create --print-join-command
Create the worker Nodes and Join the worker nodes to the control plane with the below command:-
===================================================
Run all the previous Control Plane installation commands and then run the below join command
kubeadm join 172.31.46.120:6443 --token ofefe4.7qwt2h0wbn7jfpg1 --discovery-token-ca-cert-hash sha256:b89d4d35a6ac54d616c1cf7dc26c807bedbacf17dc11ecf4686a2e6ae29868d3
After the installation verify the cluster
====================================================
kubectl get nodes
