An agent that collects metrics from an zettelkasten and stores them into an InfluxDB bucket.
Note
This is still a work in progress. If you find any bugs or have any questions, feel free to open an issue.
- Collect metrics from a local directory or a git repository
- Backfill historical metrics using git
- Parses both markdown and wiki links
- Authenticate in private git repositories using personal access tokens
- Grafana dashboards included
The exporter is distributed as both a binary and a Docker image. The currently documented ways of deploy are via docker compose and Kubernetes. For details about the setup, check out the examples:
Note that for a complete solution, it will be necessary to setup InfluxDB and Grafana. For more information about setting them up, refer to their documentation. Here are some links that might be useful:
- https://grafana.com/docs/grafana/latest/getting-started/get-started-grafana-influxdb/
- https://docs.influxdata.com/influxdb/v2/get-started/setup/
The provided dashboard uses Flux as the language to query InfluxDB, so make sure to set the "Query language" option to "Flux" when creating the InfluxDB data source in Grafana.
All configuration is supplied via environment variables. You should supply at least the required variables and either the ZETTELKASTEN_DIRECTORY or the ZETTELKASTEN_GIT_URL variables.
| Name | Description | Default | Required |
|---|---|---|---|
| INFLUXDB_URL | The InfluxDB URL | Yes | |
| INFLUXDB_TOKEN | The InfluxDB token to authenticate in the bucket | Yes | |
| INFLUXDB_ORG | The InfluxDB org containing the bucket | Yes | |
| INFLUXDB_BUCKET | The InfluxDB bucket to register metrics | Yes | |
| ZETTELKASTEN_DIRECTORY | The local directory containing the zettelkasten | No | |
| ZETTELKASTEN_GIT_URL | The URL for the git repository containing the zettelkasten | No | |
| ZETTELKASTEN_GIT_TOKEN | The access token to authenticate with private repositories | No | |
| ZETTELKASTEN_GIT_BRANCH | The branch to use for git repositories | main | No |
| COLLECTION_INTERVAL | Time to wait between metric collections | 5m | No |
| COLLECT_HISTORICAL_METRICS | Wether to collect historical metrics at startup | true | No |
| IGNORE_FILES | Comma separated list of files that will be ignored in the collection | .git,obsidian,.trash,README.md | No |
| LOG_LEVEL | The minimum log level | INFO | No |
The exporter collects metrics by parsing the contents of the markdown files present in the Zettelkasten. Currently the exporter stores metrics for individual notes and also aggregated metrics describing the entire Zettelkasten. The combination of raw and pre processed metrics allows for both flexibility and efficiency when querying the data, at the cost of a slightly higher storage usage. The two sets of metrics are stored in the same InfluxDB bucket under different measurement names.
The following table describes all metrics collected by the exporter and their respective measurement names:
| Measurement | Name | Description |
|---|---|---|
| notes | link_count | Number of links in the note |
| notes | word_count | Number of words in the note |
| notes | backlink_count | Number of links that reference the note |
| total | note_count | Number of notes in the Zettelkasten |
| total | link_count | Number of links in the Zettelkasten |
| total | word_count | Number of words in the Zettelkasten |
These are some features that I'd like to include in the future.
- Collect additional metrics
- Support Prometheus remote write as a storage
- Support VictoriaMetrics as a storage
https://prometheus.io/docs/instrumenting/writing_exporters/
https://medium.com/tlvince/prometheus-backfilling-a92573eb712c
https://github.com/influxdata/helm-charts/tree/master/charts/influxdb2
https://grafana.com/docs/grafana/latest/getting-started/get-started-grafana-influxdb/
https://docs.influxdata.com/flux/v0/get-started/
https://github.com/onedr0p/exportar
https://github.com/mischavandenburg/zettelkasten-tracker