Home
FOSSology generates a large set of data that is exported to the time-series influx database and visualized with the help of Grafana. I Wrote a fossdash_publisher script that collects useful data from FOSSology DB (Postgres) and exposes them to influx DB. Developed a visualization dashboard in Grafana by integrating influx as an input data source.
This project is divided into two parts:
- Generating meaningful data from fossology DB and publish those data to InfluxDB ( Time-series database ) using a fossdash-publisher script.
- In the Grafana using a query tool, We can get InfluxDB data and show it using meaningful charts and graphs.
Fossdash-setup---Getting-started
- Wrote a
fossdash-publisherscript in python to get the latest data from Postgres and convert them to InfluxDB standard formate and then push them to InfluxDB.install/fossdash/fossdash-publish.py.in- Metric-reported config for fossdash:
install/fossdash/fossdash_metrics.yml
- changes in respective
Makefile - Added fossdash configuration.
src/lib/php/fossdash-config.phpsrc/www/ui/admin-fossdash-config.php
- Changes in fossology sysconfig UI to add Bootstrap classes.
src/www/ui/admin-config.phpsrc/lib/php/common-sysconfig.php
- Developed a Dashboard in grafana to show these data metrics in a meaningful way.
- Generic Dashboard: It shows combined information of all instances of fossology.
- Instance-specific-Dashboard: It shows only instance-specific data metric on the dashboard. You can choose any specific instance name from the dropdown.
- Checking cron job entry using :
sudo -u www-data crontab -lOR check file/var/spool/cron/crontabs/www-data(in Docker, in the Web container)
-
GitHub Repo (Publisher script):
- My pull request:
- My local working branch: https://github.com/darshank15/fossology/tree/fossdash_gsoc2020
-
Github Repo (InfluxDB and Grafana Dashboard): https://github.com/Orange-OpenSource/fossdash
- Accepted & Merged Pull Request:
- Student: Darshan Kansagara (darshank15)
- The Project Mentor : Nicolas Toussaint (NicolasToussaint)
- Mentor(s): Gaurav Mishra (@GMishx), Shaheem Azmal (@shaheemazmalmmd), Sandipbhuyan (@sandipbhuyan)
| Tasks | Planned | Completed |
|---|---|---|
| Creating fossdash-publisher script | Yes | ✔️ |
| Integration with InlfuxDB and Grafana | Yes | ✔️ |
| Creating Dashboard on Grafana | Yes | ✔️ |
| Configuring fossdash from fossology | Yes | ✔️ |
| Handling report and unreported files on disk | Yes | ✔️ |
- Planning to extract more useful metrics from fossologyDB.
- Creating a new dashboard for new metrics.
- Adding Log counter into Dashboard: such as count of error, fatal, warning in fossology.
- Maintaining fossdash repo and planning to add source installation for different OS.
- Continue contributing to FOSSology.
- Got a deep drive knowledge on Python.
- Learned about Time-series databases such as influxDB and Prometheus and their client library.
- Learned Grafana, its Configuration, setup data source, plugging, Creating dashboard.
- Learned PHP language. I used PHP first time in this project and learned to code from this project.
- Sharpen my skills on various GIT commands such as rebase, revert, reset and pull request.
- Learned to work in Linux based environment and learned more on useful commands such as grep, find and crontab etc..
- Learned the importance of time management when working in a collaborative team to deliver milestones.
- Improved my communication & presentation skill