Automating Microservices Deployment on AWS EKS with Jenkins CI/CD

Workflow Overview:

Code Repository Management:
- Structured a multibranch Git repository, with each branch dedicated to a specific microservice, ensuring clear separation and management of individual service codebases.
Automated Environment Setup:
- Configured an EC2 instance using an automated shell script, installing essential tools such as AWS CLI, kubectl, eksctl, Docker, and Java JDK 17, to standardize the development environment.
EKS Cluster Creation:
- Established a robust Amazon EKS cluster to orchestrate containerized applications.
- Deployed Jenkins on the EC2 instance to facilitate continuous integration and continuous deployment (CI/CD) processes.
Kubernetes Configuration:
- Implemented Kubernetes service accounts, roles, and secrets within the EKS cluster to manage access controls and secure deployments.
Credential Management:
- Seamlessly integrated Jenkins with necessary credentials, including Docker registry, GitHub access tokens, and Kubernetes tokens, to enable secure and efficient communication between services.
Continuous Integration Pipeline:
- Developed a multibranch CI pipeline in Jenkins, utilizing webhooks to automate build triggers upon code commits, ensuring timely integration and testing of changes.
Continuous Deployment Pipeline:
- Authored and deployed Jenkins file for each microservice, automating the deployment process.
- Configured the CD pipeline to auto-trigger upon successful builds, streamlining the deployment workflow and ensuring rapid delivery of updates.

Commands:

Jenkins admin password retrieval on EC2 machine:

sudo cat /Users/Shared/Jenkins/Home/secrets/initialAdminPassword

To install Jenkins on Linux machine:

https://www.jenkins.io/doc/book/installing/linux/

To install AWS CLI, Kubectl, Eksctl on our Ubuntu machine run this script:

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
sudo apt install unzip
unzip awscliv2.zip
sudo ./aws/install

curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.19.6/2021-01-05/bin/linux/amd64/kubectl
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin
kubectl version --short --client

curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin
eksctl version

For best results, create a new file for script(.sh), paste the above commands in it and then make the file executable using the command: sudo chmod +x fileName.sh. Finally run it using ./fileName.sh

Next step is to run:

aws configure

Configure AWS using your AWS access key and AWS Secret Access key. Choose the default region nearest to you.

Create EKS cluster:

eksctl create cluster --name=EKS-Cluster \
                      --region=us-east-1 \
                      --zones=us-east-1a,us-east-1b \
                      --without-nodegroup

Here, you will need to define the name for your cluster in the name parameter and specify the region and zones for the cluster in the region and zones parameters, respectively.

Set up the IAM OIDC provider:

eksctl utils associate-iam-oidc-provider \
    --region us-east-1 \
    --cluster EKS-Cluster \
    --approve

IAM OIDC stands for IAM Open id connect.

This help service account which will be created within eks cluster assume IAM rules.

Create nodegroup:

eksctl create nodegroup --cluster=EKS-Cluster \
                       --region=us-east-1 \
                       --name=node2 \
                       --node-type=t3.medium \
                       --nodes=3 \
                       --nodes-min=2 \
                       --nodes-max=4 \
                       --node-volume-size=20 \
                       --ssh-access \
                       --ssh-public-key=DevOps \
                       --managed \
                       --asg-access \
                       --external-dns-access \
                       --full-ecr-access \
                       --appmesh-access \
                       --alb-ingress-access

We are defining the node name in the name parameter, and the node type, which is the machine size (here it is t3.medium). The nodes parameter defines how many worker nodes will be running. The nodes-min and nodes-max parameters define the minimum and maximum number of nodes for auto-scaling as per requirements. We have to define our public key in the ssh-public-key parameter.