A bundled Prometheus collector and exporter of Phobos metrics.
Exporter is a simple Sinatra app which can be mounted in eg a Rack App.
Collector initializes Prometheus metrics and sets up a subscribtion to certain Phobos events to keep monitor of your metrics
Add this line to your application's Gemfile:
gem 'phobos_prometheus'
And then execute:
$ bundle
Or install it yourself as:
$ gem install phobos_prometheus
Step 1: In phobos_boot.rb, configure the library by calling PhobosPrometheus.configure
with
the path of your configuration file. Note that PhobosPrometheus expects Phobos.configure to have
been run since it is using Phobos.logger
PhobosPrometheus.configure('config/phobos_prometheus.yml')
Step 2: In phobos_boot.rb, add PhobosPrometheus.subscribe
to setup tracking of Phobos metrics.
Step 3: In config.ru, mount the metrics endpoint:
run Rack::URLMap.new(
'/metrics' => PhobosPrometheus::Exporter,
# ...
)
There are three major keys to consider: counters
, histograms
and buckets
. You probably also
want to update metrics_prefix
to differentiate between different consumer apps.
For a list of possible instrumentation events, see Phobos and PhobosDBCheckpoint.
The counters
section provides a list of instrumentation labels that you want to create counters
for. For example, in order to count the number of processed events:
counters:
- instrumentation: listener.process_message
The histograms
section provides a list of instrumentation labels that you want to create
histograms for. Histograms are a bit more complex as they require bin sizes, these can be named and referenced via bucket_name
For example, in order to count the duration of processed events:
histograms:
- instrumentation: listener.process_message
bucket_name: message
The example above assumes you have defined a bucket with name message
, see below.
The buckets
section provides a definition of bucket sizes having named labels that you need to
reference for configuring histograms.
To connect with the bucket example above, we need to create a bucket named message
e.g:
buckets:
- name: message
bins:
- 5
- 10
- 25
# - ...
The gauges
section provides a list of bi-directional gauges. The provided label
will be used as the prometheus label, and a counter with this label will be incremented on any event matching the instrumentation label given in increment
and decremented by events matching decrement
.
In order to count the number of active handlers, one could do this:
gauges:
- label: number_of_handlers
increment: listener.start_handler
decrement: listener.stop_handler
This section provides some inspiration to get your Grafana dashboards up and running.
The expressions are recommended to be added as Prometheus rules, providing views that grafana can use. But you can use the expressions directly in Grafana as well.
Total processed messages per 1min:
record: <<your app name here>>_process_message_total:per_min
expr: (sum by(job, handler, group_id) (increase(<<your app name here>>_listener_process_message_total[1m])))
Total processed batches per 1min:
record: account:overview_consumer_process_batch_total:per_min
expr: (sum by(job, handler, group_id) (increase(<<your app name here>>_listener_process_batch_total[1m])))
99th percentile of process message duration:
record: account:overview_consumer_process_message_duration:p99
expr: (histogram_quantile(0.99, sum by(job, handler, group_id, le) (increase(<<your app name here>>_listener_process_message_duration_bucket[5m]))))
After checking out the repo, run bundle install
to install dependencies. Then, run rake spec
to
run the tests. You can also run bin/console
for an interactive prompt that will allow you to
experiment.
To install this gem onto your local machine, run bundle exec rake install
. To release a new
version, update the version number in version.rb
, and then run bundle exec rake release
, which
will create a git tag for the version, push git commits and tags, and push the .gem
file to
rubygems.org.
Bug reports and pull requests are welcome on GitHub at https://github.com/phobos/phobos_prometheus.