Hardik Gandhi's Blog

Automating Kubernetes Deployments with GitOps and ArgoCD on EC2

Hardik Gandhi's photo
Hardik Gandhi
·

4 min read

Introduction:

In today's fast-paced development environment, managing Kubernetes deployments efficiently is crucial. By leveraging GitOps principles with tools like ArgoCD, we can automate and streamline the deployment process. This blog explores how we implemented an automated deployment pipeline for a three-tier application using GitOps on EC2.

Project Overview:

Our project aimed to create a seamless deployment pipeline using:

  • ArgoCD for continuous integration and delivery

  • Kubernetes deployed on EC2 using KIND

  • Prometheus and Grafana for monitoring, configured via Helm

  • A front-end web app in Python which lets you vote between two options for the ultimate question: Which animal you prefer?

    • Cats

    • Dogs

  • A Redis which collects new votes

  • A .NET worker which consumes votes and stores them in…

  • A Postgres database backed by a Docker volume

  • A Node.js web app which shows the results of the voting in real

Project walkthrough:

  • Launch an AWS EC2 instance.

  • Install Docker and Kind.

  • Create a Kubernetes cluster using Kind.

  • Install and access Kubectl.

  • Install and configure Argo CD.

  • Setup the app with Argo CD.

  • Connect and manage your Kubernetes cluster with Argo CD.

  • Set Up Prometheus & Grafana.

Create Instance:

The instance should be atleast of t2.medium size.

Connect to your ec2 using ssh or any other method you prefer.

Update system:

sudo apt update

Install Docker

sudo apt-get install docker.io

Give required permission:

sudo usermod -aG docker $USER && newgrp docker

Creating and Managing Kubernetes Clusters with Kind

Create a 3-node Kubernetes cluster using Kind(Kubernetes IN Docker):

kind create cluster --config=config.yml --name=my-cluster

Check cluster information:

kubectl cluster-info --context kind-kind
kubectl get nodes
kind get clusters

Installing kubectl

curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.19.6/2021-01-05/bin/linux/amd64/kubectl
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin
kubectl version --short --client

Managing Docker and Kubernetes Pods

Check Docker containers running:

docker ps

Cloning and Running the Example Voting App

Clone the voting app repository:

git clone https://github.com/hardikgandhi03/k8s-kind-voting-app.git
cd example-voting-app/

Installing Argo CD

Create a namespace for Argo CD:

kubectl create namespace argocd

Apply the Argo CD manifest:

kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

Check services in Argo CD namespace:

kubectl get svc -n argocd

Expose Argo CD server using NodePort:

kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "NodePort"}}'

Forward ports to access Argo CD server:

kubectl port-forward -n argocd service/argocd-server 8443:443 &

Add port 8443 in security group for EC2 instance.

Access the Argo CD:

Open public address of your ec2 machine and give port 8443.

Default username is: admin

Argo CD Initial Admin Password

Retrieve Argo CD admin password:

paste the command in terminal and you will get the password.

kubectl get secret -n argocd argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d && echo

Check all pods that are running:

kubectl get all

Forward local ports for accessing the voting and result apps:

kubectl port-forward service/vote 5000:5000 --address=0.0.0.0 &
kubectl port-forward service/result 5001:5001 --address=0.0.0.0 &

Do not forget to add 5000 and 5001 in security group!

Website:

Result:

Install HELM

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

Install Kube Prometheus Stack

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo add stable https://charts.helm.sh/stable
helm repo update
kubectl create namespace monitoring
helm install kind-prometheus prometheus-community/kube-prometheus-stack --namespace monitoring --set prometheus.service.nodePort=30000 --set prometheus.service.type=NodePort --set grafana.service.nodePort=31000 --set grafana.service.type=NodePort --set alertmanager.service.nodePort=32000 --set alertmanager.service.type=NodePort --set prometheus-node-exporter.service.nodePort=32001 --set prometheus-node-exporter.service.type=NodePort

Do port forwarding for Prometheus and Grafana.

kubectl port-forward svc/kind-prometheus-kube-prome-prometheus -n monitoring 9090:9090 --address=0.0.0.0 &
kubectl port-forward svc/kind-prometheus-grafana -n monitoring 3000:80 --address=0.0.0.0 &

Add 9090 and 3000 in security group.

Prometheus:

Some Prometheus Queries

sum (rate (container_cpu_usage_seconds_total{namespace="default"}[1m])) / sum (machine_cpu_cores) * 100

sum (container_memory_usage_bytes{namespace="default"}) by (pod)


sum(rate(container_network_receive_bytes_total{namespace="default"}[5m])) by (pod)
sum(rate(container_network_transmit_bytes_total{namespace="default"}[5m])) by (pod)

Grafana Dashboard

Grafana Monitoring

Deleting Kubernetes Cluster

kind delete cluster --name=kind

Delete the EC2 instance that you have created.

Thank you so much for reading till the end :)

DevopsArgoCDKubernetes#prometheusGrafanamonitoring