Cloud Native Orchestration

Capstone project for Udacity's Cloud DevOps Engineer

✔️ Description

This project consists of deploying an API on Kubernetes using deployment Rolling Update strategy.
The application used in this project is based on Python and Flask. This project has a single endpoint:

• /api/v0/multiply
• Usage: http://localhost:5001/api/v0/multiply?param1=12&param2=2

✔️ Content of Repository

• 📁 Folders
  • 📄 pipelines - Definitions of the pipeline used by Jenkins
  • 📄 descriptors-k8s - Descriptors to deploy application in the Kubernetes
  • 📄 images - Evidences of pipeline operation
• 📄 Dockerfile - Docker descriptor
• 📄 main.py - Web application
• 📄 requirements.txt - Definitions of dependencies of the application
• 📄 run_docker.sh - Script to build docker image

✔️ Preparing CI/CD Environment

It was used Jenkins to automate manage of the Cluster Kubernetes and Application on AWS EC2 Instance based on Ubuntu 18.04 image.

Requirements:

• Java
  • $ sudo apt-get update
  • $ sudo apt install -y default-jdk
• Jenkins
  • $ wget -q -O - https://pkg.jenkins.io/debian/jenkins-ci.org.key | sudo apt-key add -
  • $ sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
  • $ sudo apt-get update
  • $ sudo apt-get install jenkins
  • $ sudo systemctl start jenkins
  • $ sudo systemctl enable jenkins
  • $ sudo systemctl status jenkins
• Jenkins Plugins
  • Blue Ocean
  • Pipeline: AWS Steps
  • CloudBees AWS Credentials
• Python and other tools
  • $ sudo apt install python3
  • $ sudo apt install python3-pip
  • $ sudo apt install virtualenv
  • $ sudo pip install pylint && pip install pylint --upgrade
  • $sudo cp /home//.local/bin/pylint /usr/local/bin
• Pip for Python
  • $ sudo apt install python-pip
• AWS Client V2.0
  • $ curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
  • $ unzip awscliv2.zip
  • $ sudo ./aws/install
• EKSCTL
  • $ curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
  • $ sudo mv /tmp/eksctl /usr/local/bin
• AWS-IAM-Authenticator
  • $ curl -o aws-iam-authenticator https://amazon-eks.s3.us-west-2.amazonaws.com/1.15.10/2020-02-22/bin/linux/amd64/aws-iam-authenticator
  • $ chmod +x ./aws-iam-authenticator
  • $ sudo mv aws-iam-authenticator /usr/local/bin
• Docker Engine
  • $ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
  • $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
  • $ sudo apt-get update
  • $ sudo apt-get install docker-ce docker-ce-cli containerd.io
  • $ sudo chmod 777 /var/run/docker.sock
• Hadolint
  • $ wget https://github.com/hadolint/hadolint/releases/download/v1.1/hadolint_linux_amd64
  • $ sudo chmod +x hadolint_linux_amd64
  • $ sudo mv hadolint_linux_amd64 hadolint
  • $ sudo mv hadolint /usr/local/bin
• Kubectl
  • $ curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.18.0/bin/linux/amd64/kubectl
  • $ chmod +x ./kubectl
  • $ sudo mv ./kubectl /usr/local/bin/kubectl
• Shopfy/krane
  • $ sudo apt install ruby-full
  • $ sudo gem install rake
  • $ sudo gem install rainbow -v '2.2.2'
  • $ sudo gem install krane

💡 Tip - Use AIM Role:
Create a AIM Role with all policies AmazonEKS* and attach it on EC2 Instance running Jenkins. This way you don't need to configure your credentials into EC2 instance.

✔️ Validating Environment

1 - Connect to EC2 instance and execute the command below to create a cluster with 3 nodes. This command can to take about 15 minutes to finish. Be patient!
$ eksctl create cluster --name <cluster-name> --region <region> --nodegroup-name standard-workers --node-type t3.medium --nodes 3 --nodes-min 1 --nodes-max 4 --managed

2 - Next, we need to update ˜/.kube/config file, so that you can use the kubectl command.
$ aws eks update-kubeconfig --name <cluster-name> --region <region>

3 - Now, we let's get information of the cluster using kubectl command.
$ kubectl cluster-info

4 - Finally, delete cluster.
$ eksctl delete cluster cluster --name <cluster-name> --region <region>

✔️ Rolling Update Release

As defined by the website kubernetes.io, rolling updates is a strategy that allow you deployments' update to take place with zero downtime by incrementally updating Pods instances with new ones.
Below snnipet code from the file deployment.yaml used in this project to deploy application.

...
spec:
  replicas: 2
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 50%
      maxSurge: 1
...

✔️ References

• Getting started with EKSCTL - https://docs.aws.amazon.com/eks/latest/userguide/getting-started-eksctl.html
• Launch a guest book application - https://docs.aws.amazon.com/eks/latest/userguide/eks-guestbook.html
• Create a SSH Key - https://eksworkshop.com/prerequisites/sshkey.html
• Install Jenkins on Ubuntu - https://wiki.jenkins.io/display/JENKINS/Installing+Jenkins+on+Ubuntu
• How to Install Pip on Ubuntu 18.04 - https://linuxize.com/post/how-to-install-pip-on-ubuntu-18.04/
• Install Docker Engine on Ubuntu - https://docs.docker.com/engine/install/ubuntu/
• Installing aws-iam-authenticator - https://docs.aws.amazon.com/eks/latest/userguide/install-aws-iam-authenticator.html
• Install and Set Up kubectl - https://kubernetes.io/docs/tasks/tools/install-kubectl/
• Shopfy/krane - https://github.com/Shopify/krane
• Hadolint v1.1 - https://github.com/hadolint/hadolint/releases/tag/v1.1