-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update manifests to latest version of Prometheus Operator #16
Conversation
These annotations made sense in pre v1.3.0 Prometheus releases, however, with >=v1.3.0 and the Prometheus Operator these annotations are more confusing then helpful.
The latest version of the Prometheus Operator requires Prometheus >=v1.4.0 for the Alertmanger discovery feature.
The ports reflect the upstream kube-dns manifests of bootkube and kubernetes/kubernetes.
👍 looks good. Yes, we should cut another version to integrate into Tectonic etc. |
The annotation is not supported by the operator. ServiceMonitors should be
used instead. The annotation is very limited by its nature and there's no
feasible way to support anything beyond "scrape on" and a single port. A
per-service scrape interval, multi-port pods, authentication, honorLabels,
and more are all not possible. It also provides no meaningful way to assign
services to be scraped by different Prometheus instances.
ServiceMonitors allow all these without exposing any complex relabelling
and there's really no practical reason for sticking with the annotation.
…On Fri, Jun 2, 2017 at 2:26 PM Julian V. Modesto ***@***.***> wrote:
@brancz <https://github.com/brancz> sorry if this isn't the best place to
ask!
Regarding 0c1bb5c
<0c1bb5c>,
why were prometheus.io/scrape annotations removed? Is there any way to
configure Prometheus Operator to discover pods to scrape based on these
annotations still?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#16 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AEuA8ncH_YnOGepPiKJVVzmgcMQwTWd5ks5sAH3vgaJpZM4LCl_q>
.
|
Got it – this is helpful and makes sense after consuming, thank you! |
The problem I see here is that this annotations are fully self service and a well working way to communicate the metrics that should be scraped to prometheus. With ServiceMonitors this self service is not working anymore as they need to be in the monitoring namespace and I don't want to give everybody access to it. |
The ServiceMonitor has a namespaceselector for the services, so the only thing you to do is have a convention in your organization, which boils down to the same as the annotation. You simply need you organization to have a commonly named port and a label like "monitoring: true". The namespace selector can then select Services globally. |
However, we acknowledge the want that people have, and are trying to think of a way we can introduce something more broad without violating the concerns we currently have. |
We have an established convention, but also often have third party tools installed which have their own conflicting conventions. As a hybrid solution, we are applying a custom prometheus on top of the operator. Thanks again @solsson |
I'd like to add one more thing to this issue. |
That is exactly our issue. 3rd party helm apps often have this need, which is in conflict with our using the operator at present. |
I found following solution.
Then all pods with prometheus.io/scrape annotations are processed. |
@maver1ck Could you please tell me where I should put this configuration ? |
You need to add this to helm chart configuration (values.yaml) Unfortunately this option is undocumented. |
There is also some information available at https://github.com/coreos/prometheus-operator/blob/master/Documentation/additional-scrape-config.md |
you can also add this config to scrape services
|
* Make eventing controller observable - Enable scrapping of eventing-controller - Add eventing reconciler dasshboard * Retain vendor license * Removed eventing reconciler dashboard * Make broker filter and ingress scrapeable * Removed hard coded annotation for prometheus - Refer prometheus-operator/kube-prometheus#16 * Made sources-controller observable * Remove sources metrics service * Fixed as per PR comments
* Make eventing controller observable - Enable scrapping of eventing-controller - Add eventing reconciler dasshboard * Retain vendor license * Removed eventing reconciler dashboard * Make broker filter and ingress scrapeable * Removed hard coded annotation for prometheus - Refer prometheus-operator/kube-prometheus#16 * Made sources-controller observable * Remove sources metrics service * Fixed as per PR comments
Update about |
If you're compiling the yamls using the jsonnet file. Follow the instructions mentioned earlier to create the additional scrape config secret. I tried to patch the prometheus resource directly after building/applying the yamls via jsonnet. It didn't work. I had to add the This does not go in the prometheus+:: {
prometheus+: {
spec+: {
additionalScrapeConfigs+: {
name: 'additional-scrape-configs',
key: 'additonal-scrape-configs.yaml'
},
},
},
}, |
Tried out with a container image of
HEAD
of the Prometheus Operator.List of changes divided into commits:
node-exporter
versionIt would probably make sense to cut another release of the Prometheus Operator, just so we can at least partially version this until we get versioned user-guides.
@fabxc