This repo contains the step by step implementation of SRE Bootcamp and updated README for the steps I followed to completed each milestone🏅
- OS: Ubuntu 22.04 LTS
- Web server: Node.js, Expressjs
- Unit Tests: Jest,Supertest and Postman
- Database: PostgreSQL
- Database Migration Tool: Knex
- Nodejs version: v20.14.0 LTS
- Containerisation: Docker and Docker-Compose
- CI: Github Actions
- Container Orchestration Tool: Kubernetes
- Load Balancer: Nginx
- External Secret Store: AWS Secrets Manager
- K8s Custom Resource Definitions: External Secrrets
- Create a simple REST API Webserver
- Containerise REST API
- Setup one-click local development setup
- Setup a CI pipeline
- Deploy REST API & its dependent services on bare metal
- Setup Kubernetes cluster
- Deploy REST API & its dependent services in K8s
- Deploy REST API & its dependent services using Helm Charts
- Setup one-click deployments using ArgoCD
- Setup an observability stack
- Configure dashboards & alerts
- Once you have an Ubuntu system follow the below steps assuming you already have git CLI installed
sudo -i // switch to root user
apt update -y
- Clone the repo:
git clone https://github.com/rohit1101/SRE-Bootcamp-Web-Server.git
- Install the dependencies required for the project:
# installs nvm (Node Version Manager)
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
# download and install Node.js
nvm install 20
# verifies the right Node.js version is in the environment
node -v # should print `v20.14.0`
# verifies the right NPM version is in the environment
npm -v # should print `10.7.0`
- Install postgresql:
apt install postgresql -y // this install psql client for interacting with the database with queries
-
Run the following command to set
postgres
user with a valid password and exit psql client using\q
and typeexit
to logout frompostgres
user shell:
ALTER USER postgres PASSWORD 'postgres';
After successful pre-requisites setup following the below steps:
- Now let us clone this repo(fork and clone) and move into the src of the web server code:
git clone https://github.com/rohit1101/SRE-Bootcamp-Web-Server.git
cd SRE-Bootcamp-Web-Server
- Execute
make install
to install all the dependancies for the express js web server - Run
make db_config
command creates aknexfile.js
configuration file. - Create a new directory named migrations
mkdir migrations
- Make sure you pass the correct values for the environment variables in a new file named
.env
by referring the.env.example
. My.env
file looks as shown below.*Since I am testing the API locally I am using the default postgres DB for creating tables(on a production environment this is not recommended).
NODE_ENV=development
DB_USER=postgres
DB_PASSWORD=postgres
DB_HOST=127.0.0.1
DB_PORT=5432
DB_DATABASE=postgres
- Modify the
knexfile.js
configuration file based on your requirements
require("dotenv").config({ path: ".env" });
module.exports = {
development: {
client: "pg",
connection: {
user: process.env.DB_USER,
host: process.env.DB_HOST,
database: process.env.DB_DATABASE,
password: process.env.DB_PASSWORD,
},
pool: {
min: 2,
max: 10,
},
migrations: {
directory: "./migrations",
},
},
};
- Create a new migration, the following command will create a new migrations file in this path ->
migrations/
and update the migrations file by refering the migrations file in this repo.
make create_migrations
- This command applies the migration and creates the students table in your PostgreSQL database.
make migrate
![image](https://private-user-images.githubusercontent.com/37110560/336802462-304fa9f3-8a78-4b7c-b7cc-2722a3adefd0.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU2MDI1OTEsIm5iZiI6MTcyNTYwMjI5MSwicGF0aCI6Ii8zNzExMDU2MC8zMzY4MDI0NjItMzA0ZmE5ZjMtOGE3OC00YjdjLWI3Y2MtMjcyMmEzYWRlZmQwLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MDYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTA2VDA1NTgxMVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTQ0ZDRlMTgwNWRmYmMyM2U5NmUyNTljMDYyYzUzN2Q2NjdhZGYxZWJiMjFjMTM0NzRjM2VkNjBmYjg0OGRmYWUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.DCb8CC7pFKVMXwvRGwx-pNKLm7FUHM1J_G19WfY3tqg)
NOTE: Link for postman collection
Base URL -> http://locahost:3000/v1
- Description: Checks API health status.
- Response:
200 OK
Example Request:
curl -X GET http://localhost:3000/v1/healthcheck
- Description: Retrieves all students.
- Response:
200 OK
Example Request:
curl -X GET http://localhost:3000/v1/students
- Description: Retrieves student by ID.
- Response:
200 OK
/404 Not Found
Example Request:
curl -X GET http://localhost:3000/v1/students/1
- Description: Creates a new student.
- Response:
200 Created
/400 Bad Request
- Request body:
- name:
string
- age:
number
- department:
string
- name:
Example Request:
curl -X POST http://localhost:3000/v1/students \
-H "Content-Type: application/json" \
-d '{"name": "Alice", "age": 23, "department": "MECH"}'
- Description: Deletes student by ID.
- Response:
200 No Content
/404 Not Found
Example Request:
curl -X DELETE http://localhost:3000/v1/students/4
- Description: Updates student by ID.
- Response:
200 OK
/400 Bad Request
/404 Not Found
- Request body:
- name:
string
- age:
number
- department:
string
- name:
Example Request:
curl -X PUT http://localhost:3000/v1/students/5 \
-H "Content-Type: application/json" \
-d '{"name": "Robert", "age": 24, "department": "ECE"}'
Follow the instructions to install docker on your Ubuntu Machine -> Install docker on ubuntu
- Create a Dockerfile for running the web server using
node-*-apline
image for reducing the image size
ARG NODE_VERSION=20.14.0
FROM node:${NODE_VERSION}-alpine AS build
# Set working directory as /app
WORKDIR /app
# copy deps
COPY ./package*.json ./
# install the dependencies
RUN npm install
# copy all source code
COPY . .
FROM build as run
WORKDIR /app
COPY --from=build /app ./
# set envs required for the web-server to run
ARG NODE_ENV=development
ARG DB_USER=postgres
ARG DB_PASSWORD=postgres
ARG DB_HOST=127.0.0.1
ARG DB_PORT=5432
ARG DB_DATABASE=postgres
# expose port 3000 for the web server to be accessed
EXPOSE 3000
# execute this following command once the container boots
ENTRYPOINT ["npm","start"]
- Command for building a Docker Image and pushing it to Docker Hub (This step requires you to have a Docker Hub account):
# Create an image
docker build -t <docker-hub-repo-name>/<image-name>:<tag-name>
docker push <docker-hub-repo-name>/<image-name>:<tag-name>v1.0
- To execute
make containerise
to pull the image from docker registry and create aweb-server
container make sure your provide your Docker Hub repo name and image with the desired tag. Please refer the command here - NOTE: In the above Dockerfile I have directly passed a environment variables but in production environment it is better to use Docker secrets.
Now let's make the local developement setup quicker by using docker-compose
and trigger server and DB containers using a single make
command
-
First let us update the
.env
file with appropriate values since when we create containers usingdocker-compose
NODE_ENV=development DB_USER=postgres DB_PASSWORD=postgres DB_HOST=db DB_PORT=5432 DB_DATABASE=postgres
-
Please refer the
compose.yaml
here for creating DB container first and checking the health of postgresql DB which then triggers migrations and starts the web-server. -
The following is the updated Dockerfile for applying migrations and starting the web server
ENTRYPOINT ["sh", "-c", "npx knex migrate:latest && npm start"]
-
Run
make start_app
for starting the app (web-server + db) -
Make a the following
curl
request and openhttp://localhost:3000/v1/students
to view all students as response
curl -X POST http://localhost:3000/v1/students \
-H "Content-Type: application/json" \
-d '{"name": "Alice", "age": 23, "department": "Arts"}'
- Run
make delete_app
for stopping the app (web-server + db) - NOTE: On ubuntu use
docker compose
plugin insteaddocker-compose
- For setting up self-hosted Github Actions Runner we need to navigate to the Settings tab from the repository.
- Then click on Actions dropdown and click on Runners.
- Click on New self-hosted runner button
- Follow the instructions on this page and setup a self-hosted runner and make sure to set
runs-on: self-hosted
- Finally let us start our self-hosted runner to run workflows on our system:
cd actions-runner/ ./run.sh
- To setup github actions follow the below command from the root of the project folder and write your github action config on a
.yaml
filemkdir -p .github/workflows/ cd .github/workflows/ touch <github-action-filename>.yaml
For this Milestone, I am using VirtuaBox for using Vagrant for Virtualisation Install VirtualBox on a desired OS -> Install VirtualBox
Steps for installing Vagrant on a desired OS -> Install Vagrant
- First let us setup the following architecture using docker-compose on host machine and test it.
![image](https://private-user-images.githubusercontent.com/37110560/338452117-3ff2fed6-97ee-4056-98a0-ea0c23b9df21.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU2MDI1OTEsIm5iZiI6MTcyNTYwMjI5MSwicGF0aCI6Ii8zNzExMDU2MC8zMzg0NTIxMTctM2ZmMmZlZDYtOTdlZS00MDU2LTk4YTAtZWEwYzIzYjlkZjIxLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MDYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTA2VDA1NTgxMVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWYwNDQ0NDkxZGJlNTc1NDQ2N2E1ZjhjZmE0YzM1ZTVkNGU5YTM1MjI1OWU3OThhMWI5YWJhNjlmNGE5MmUwMzEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.ht5BDXhQZLQ4CfPmYS1tAGtT9NHH2BJuxWjVVLgw_-0)
-
nginx as load balancer by exposing
8080:80
-
ws-1 (web server 1) by exposing
8081:3000
-
ws-2 (web server 2) by exposing
8082:3000
-
db (postgres db) by exposing
5432
-
Assuming the previous step is working as expected now let us create a directory and initialise Vagrant:
mkdir <directory-name> && cd <directory-name> vagrant init hashicorp/bionic64 # creates Vagrantfile
-
Start VM using:
vagrant up
-
Login to the VM using:
vagrant ssh
-
Use
logout
to exit the box -
Create a
script.sh
file and add the installation commands for dependancies, thisscript.sh
on host machine will be synced on the VM in/vagrant/
-
Let us modify
Vagrantfile
to install the dependancies using a shell script while provisioningconfig.vm.provision "shell", inline: "/bin/bash /vagrant/script.sh", run: "once"
-
Run the following command for provisioning the VM with the shell script:
vagrant provision --provision-with shell
-
Finally let us config the port forwarding on
Vagrantfile
such that when we accesshttp://localhost:8080/v1/healthcheck
fromhost
machine the traffic is forwarded toguest
machine port8080
config.vm.network "forwarded_port", guest: 8080, host: 8080
-
Now
logout
fromguest
machine and runvagrant reload
to run our VM with the updated port forwarding setup. -
Now let us test the port forwarding:
cd SRE-Bootcamp-Web-Server/ sudo docker compose --profile DB up --build -d
-
Open your browser and test
http://localhost:8080/v1/healthcheck
NOTE: Please refer here for the script file.
- Follow this link to install
minikube
on the required OS. - Follow this link to install
kubectl
cli tool for your OS.
NOTE: Make sure docker daemon is running in background before proceeding
- Execute
minikube start --nodes 4 -p multinode-setup
to setup multi-node cluster. - Run the following command to set label for our nodes:
kubectl label node multinode-setup-m02 type=app kubectl label node multinode-setup-m03 type=db kubectl label node multinode-setup-m04 type=deps
First let us create a multinode cluster using minikube for deploying our REST APIs and the dependencies on K8s.
- Use this command to create multi-node setup with minikube:
minikube start -n=4 -p multinode-setup
- Create a namespace with
kubectl
command below:
kubectl create ns student-api-ns
- Switch to our namespace:
kubectl config set-context --current --namespace=student-api-ns
- Make sure you add your service, deployment, config maps and secrets in a single yaml file within a folder (like /app/app.yaml and /db/db.yaml)
- Go to the appropriate directory and execute which will create required object in our K8s cluster.
kubectl create -f k8s/manifest/<folder-name>/<manifest-filename>.yaml
- Use the following command to test the web server and db connection locally by port forwarding, here
30002
is thenodePort
for thews-service
and8000
is the exposed port from the pod
kubectl port-forward service/ws-service 30002:8000
- Till now we have used configMaps for passing environment variables to the respective pods, which is not the good way of doing thing in production, hence the following steps will guide you to setup external secret store and we will use AWS Secrets Manager for storing the secrets.
- Let us create a namespace for external secret store and switch the context
kubectl create ns eso-ns
kubectl config set-context --current --namespace=eso-ns
- Install external secrets operator using helm -> here
- Create config files for
SecretStore
andExternalSecretStore
- Create a secret which we can use to pass aws cli access keys to the secret store using the following command:
kubectl create secret generic awssm-secret --from-literal=access-key=<AWS_CLI_ACCESS_KEY> --from-literal=secret-access-key=<AWS_CLI_SECRET_ACCESS_KEY>
- Make sure you provide the least priviledge access to the IAM user via IAM Policy.
- Now remove all the
DB_PASSWORD
environment variables previously defined onconfigMaps
, delete the deployments and re-create the deployment to test if our changes are working. - For tearing down the minikube setup use the following command:
minikube delete --profile multinode-setup
🚧 Work in progress
🚧 Work in progress
🚧 Work in progress
🚧 Work in progress