Core Features • Frontend • Backend • Development • Testing • Deployment • Wiki • Further Improvements • Contribute
- Real-time tracking and displaying of user events (clicks and mouseover actions).
- User events information stored in database (PostgreSQL & Redis supported).
- Events stats shared via Action Cable and displayed to subscribed clients or specific clients via polling JS requests.
- Backend implementd with Ruby on Rails, Kafka and Zookeeper.
- Deployment workflow through AWS ECS script.
- Infrastructure managed through Terraform (work in progress).
- Deployment workflow through Terraform and integrated within Github Actions (CI/CD) into AWS. (work in progress)
Relevant folder: backend
$ git clone git@github.com:juanroldan1989/color-tracking.git
$ cd color-tracking
$ cd backend
$ docker-compose up
# cleaning up
$ docker-compose down -v
Open static frontend
page:
$ cd frontend
$ open index.html
Relevant folder: /backend/infrastructure/terraform
-
Enable
Docker Local Development
section inmain.tf
-
Run commands:
$ terraform init
$ terraform plan
$ terraform apply
Application is ready
Open static frontend
page:
$ cd frontend
$ open index.html
To destroy all local containers managed by your Terraform configuration:
$ terraform destroy
API tests written with RSpec
$ cd backend
$ rspec spec
Frontend work has been purposely implemented using standalone libraries:
- jQuery (Easy DOM Handling)
- D3 (Graphs)
- Action Cable (Websockets)
── frontend
└── static
├── css
│  └── main.css
├── index.html
└── js
├── draw_colors_table.js
├── events
│  └── clicks_and_hovers.js
├── polling
│  ├── draw_clicks_dashboard.js
│  └── draw_hovers_dashboard.js
└── websockets
├── action_cable.js
├── draw_clicks_dashboard.js
└── draw_hovers_dashboard.js
This way anyone can use their framework of choice to implement a frontend
application that interacts with the backend
and write frontend
tests as well.
Automated Verification achieved through Chef InSpec
Chef InSpec
is an infrastructure security and compliance testing framework with a human and machine-readable language for comparing actual versus desired system state.
Download and install Inspec: https://www.chef.io/downloads/tools/inspec
Source: https://joachim8675309.medium.com/docker-the-terraform-way-a7c16b5f59ed
Tests for project present in validate_containers_state.rb
PORT_DB = attribute("port", value: "5432", description: "DB port")
PORT_ZOO = attribute("port", value: "2181", description: "ZOOKEEPER port")
PORT_KAFKA = attribute("port", value: "9092", description: "KAFKA port")
PORT_API = attribute("port", value: "3000", description: "API port")
PORT_KARAFKA = attribute("port", value: "3001", description: "KARAFKA port")
control "db-container" do
impact 1.0
title "DB Container"
desc "DB container should be running"
describe docker_container "db" do
it { should exist }
it { should be_running }
its("image") { should eq "postgres:9.6-alpine" }
its("ports") { should eq "0.0.0.0:#{PORT_DB}->#{PORT_DB}/tcp" }
end
describe json({ command: "docker container inspect db" }) do
its([0, "HostConfig", "NetworkMode"]) { should include "color-tracking-net" }
end
end
control "zoopkeeper-container" do
impact 1.0
title "Zookeeper Container"
desc "Zookeeper container should be running"
describe docker_container "zookeeper" do
it { should exist }
it { should be_running }
its("image") { should eq "wurstmeister/zookeeper" }
its("ports") { should eq "22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:#{PORT_ZOO}->#{PORT_ZOO}/tcp" }
end
describe json({ command: "docker container inspect zookeeper" }) do
its([0, "HostConfig", "NetworkMode"]) { should include "color-tracking-net" }
end
end
control "kafka-container" do
impact 1.0
title "Kafka Container"
desc "Kafka container should be running"
describe docker_container "kafka" do
it { should exist }
it { should be_running }
its("image") { should eq "wurstmeister/kafka" }
its("ports") { should eq "0.0.0.0:#{PORT_KAFKA}->#{PORT_KAFKA}/tcp" }
end
describe json({ command: "docker container inspect kafka" }) do
its([0, "HostConfig", "NetworkMode"]) { should include "color-tracking-net" }
end
end
control "api-container" do
impact 1.0
title "API Container"
desc "API container should be running"
describe docker_container "api" do
it { should exist }
it { should be_running }
its("image") { should eq "juanroldan1989/color-tracking-api" }
its("ports") { should eq "0.0.0.0:#{PORT_API}->#{PORT_API}/tcp" }
end
describe json({ command: "docker container inspect api" }) do
its([0, "HostConfig", "NetworkMode"]) { should include "color-tracking-net" }
end
end
control "karafka-consumer-container" do
impact 1.0
title "Karafka Consumer Container"
desc "Karafka Consumer container should be running"
describe docker_container "karafka-consumer" do
it { should exist }
it { should be_running }
its("image") { should eq "juanroldan1989/color-tracking-api" }
its("ports") { should eq "0.0.0.0:#{PORT_KARAFKA}->3000/tcp" }
end
describe json({ command: "docker container inspect karafka-consumer" }) do
its([0, "HostConfig", "NetworkMode"]) { should include "color-tracking-net" }
end
end
Trigger tests:
$ cd backend
$ docker-compose up
$ cd backend/infrastructure/tests
$ inspec exec validate_containers_state.rb
[Work In Progress]
Provide Github Actions setup for triggering all frontend/backend and containers tests, THEN deploy.
-
Official steps for ECS CLI Configuration: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_CLI_installation.html
-
Place both
ecs-cli.asc
andpublic_key_ecs.txt
files insidebackend/infrastructure/aws-ecs
folder. -
Login into AWS Console -> EC2 -> Key pairs -> Create key pair
-
Download
private-ecs-key.cer
and place it insidebackend/infrastructure/aws-ecs
folder -
Deployment instructions:
backend/infrastructure/aws-ecs/setup-aws-ecs.sh
This script takes care of:
- Cluster configuration and creation
$ ecs-cli configure --region us-east-1 --cluster color-tracking --config-name color-tracking
- ECS Key Pair configuration, EC2 Instance Types and Cluster capability
$ ecs-cli up --keypair ecs --capability-iam --size 2 --instance-type t3.medium --cluster-config color-tracking --force
- ECS Deployment of tasks with containers
$ ecs-cli compose up --cluster color-tracking --create-log-groups
- Check containers status
$ ecs-cli ps --cluster color-tracking
- Cluster deletion and resources cleanup
$ ecs-cli down --cluster color-tracking --force
Relevant folder: /backend/infrastructure/terraform
-
Enable
AWS DEPLOYMENT
section inmain.tf
-
Run commands:
$ terraform init
# Observe all resources to be added
$ terraform plan
# Build & Launch infrastructure required
$ terraform apply
Application is ready
Replace http://localhost:3000
instances inside frontend/static/index.html
file with Public DNS
hostname:
-
Check AWS ECS console: https://console.aws.amazon.com/ecs/home#/clusters
-
Get
Public DNS
value provided withinECS Cluster
->Container Instance
->Public DNS
-
Open static
frontend
page:
$ cd frontend
$ open index.html
To destroy all remote objects managed by your Terraform configuration:
$ terraform destroy
Frontend work has been purposely implemented using standalone libraries:
- jQuery (Easy DOM Handling)
- D3 (Graphs)
- Action Cable (Websockets)
This way anyone can use their framework of choice to implement a frontend
application that interacts with the backend
Broadcasting Events from backend
to frontend
-
Websockets implementation in the
backend
achieved through Rails's ownActionCable
-
Websockets implementation in the
frontend
achieved throughaction_cable
Javascript library.
Main tables to interact with:
api_key
: String
id
: Integername
: String (e.g.: "blue", "red")
id
: Integername
: String (e.g.: "hover", "click")
id
: Integerapi_key
: Stringaction_id
: Integer (e.g.: "Click")color_id
: Integer (e.g.: "Red")amount
: Integer (e.g.: "30")
cable_ready
ruby gem is a great extension on ActionCable
capabilities
Provides operations
to be broadcasted to the frontend.
https://cableready.stimulusreflex.com/#what-can-i-do-with-cableready https://cableready.stimulusreflex.com/cableready-everywhere
Backend
adjustments when working with cable_ready
:
# Gemfile
...
gem "cable_ready", "~> 4.5.0"
...
# /v1/events_controller.rb
...
cable_ready[stream].console_log(message: { results: results })
cable_ready.broadcast
...
Frontend
adjustments when working with cable_ready
:
# js/websockets/draw_hovers_dashboard.js
...
console.log("Received data: ", data);
var results = data.operations.consoleLog[0].message.results;
...
Do you need some help? Check the articles from the wiki.
Got something interesting you'd like to add or change? Please feel free to Open a Pull Request
If you want to say thank you and/or support the active development of Color Tracking API
:
- Add a GitHub Star to the project.
- Tweet about the project on your Twitter.
- Write a review or tutorial on Medium, Dev.to or personal blog.