Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add errors only self-hosted infrastructure #3190

Merged
merged 11 commits into from
Jul 22, 2024

Conversation

hubertdeng123
Copy link
Member

@hubertdeng123 hubertdeng123 commented Jul 9, 2024

This does a couple of things:

Sets the default docker compose profile to "feature-complete". This way, users can opt in to using errors-only self-hosted if they want, otherwise nothing changes.

To enable errors only self hosted

Change .env file
SENTRY_SELF_HOSTED_ERRORS_ONLY = True
COMPOSE_PROFILES = errors-only

Copy link

codecov bot commented Jul 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.01%. Comparing base (ca5f35c) to head (a0333f2).

✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #3190   +/-   ##
=======================================
  Coverage   99.01%   99.01%           
=======================================
  Files           3        3           
  Lines         203      203           
=======================================
  Hits          201      201           
  Misses          2        2           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

docker-compose.yml Outdated Show resolved Hide resolved
docker-compose.yml Outdated Show resolved Hide resolved
@hubertdeng123 hubertdeng123 marked this pull request as ready for review July 16, 2024 23:15
@aldy505
Copy link
Collaborator

aldy505 commented Jul 16, 2024

Gonna review later. But we definitely need to update the docs.

Copy link
Collaborator

@aldy505 aldy505 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should also link the issue: #3113

@@ -363,3 +363,8 @@ def get_internal_network():
# https://docs.djangoproject.com/en/4.2/ref/settings/#std-setting-CSRF_TRUSTED_ORIGINS

# CSRF_TRUSTED_ORIGINS = ["https://example.com", "http://127.0.0.1:9000"]

# If you would like to use self-hosted Sentry with only errors enabled, please set this
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also define what's "errors only" and what's not? I mean it's clear you'd know what "errors only" means if you previously use Sentry. But I don't think newcomers will instantly understand about what features are available on the "feature complete" profile.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I think we should. Going to also need to update docs, which I'll do today.

Copy link

@soupdumplings soupdumplings Jul 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excuse me, does the "errors only" version include the Projects? I reviewed the docs and found that it doesn't have the Projects.

Copy link
Contributor

@stayallive stayallive Jul 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See: https://develop.sentry.dev/self-hosted/#errors-only-self-hosted-sentry. But projects are a core concept of Sentry, so yes that is always included.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. I really need this feature!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, projects are always included.

Copy link

@soupdumplings soupdumplings Jul 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://develop.sentry.dev/self-hosted/#errors-only-self-hosted-sentry
Does the "Traces" mentioned in the docs refer to "stack traces" and is it no longer supported under "errors only"?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, traces refers to the submenu inside the "Explore" menu.
image

Stack traces is included on the errors-only since it's a crucial component to the error monitoring mechanism

Copy link

@soupdumplings soupdumplings Jul 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for your patient answers. I use self-hosted Sentry as our team's crash analysis platform. It's really powerful and useful. The "errors only" version is just perfect for me.

@hubertdeng123 hubertdeng123 mentioned this pull request Jul 18, 2024
6 tasks
@@ -36,24 +36,8 @@ x-sentry-defaults: &sentry_defaults
<<: *depends_on-default
snuba-api:
<<: *depends_on-default
snuba-errors-consumer:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are not needed to bring up the web container and other ingest consumers

.env Show resolved Hide resolved
@hubertdeng123 hubertdeng123 merged commit 485d3ff into master Jul 22, 2024
14 checks passed
@hubertdeng123 hubertdeng123 deleted the hubertdeng123/errors-only-sh branch July 22, 2024 17:23
@@ -363,3 +363,6 @@ def get_internal_network():
# https://docs.djangoproject.com/en/4.2/ref/settings/#std-setting-CSRF_TRUSTED_ORIGINS

# CSRF_TRUSTED_ORIGINS = ["https://example.com", "http://127.0.0.1:9000"]

# If you would like to use self-hosted Sentry with only errors enabled, please set this
SENTRY_SELF_HOSTED_ERRORS_ONLY = env("COMPOSE_PROFILES") == "feature-complete"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be env("COMPOSE_PROFILES") == "errors-only" right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right htis is a bug, thanks for raising. I just checked my local sentry.conf.py and it uses SENTRY_SELF_HOSTED_ERRORS_ONLY = env("COMPOSE_PROFILES") != "feature-complete" when I was testing this locally. Opened #3220

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, I wasn't 100% sure but I just also debugged a self-hosted instance very bravely using nightly and was caught off guard by this 😅 Thanks for the quick PR!

@dosinabox
Copy link

Thank you for this option. But how do requirements change exactly for errors-only mode? Previously, 8 GB RAM was enough for the full set of features in version 23.7.0 (that I currently use in production), it was increased to 16 GB in version 23.11.1, and it looks like the same 16 GB is still required for the limited errors-only version 24.7.1.
I just wanted to get rid of all the features that I never use and be able to limit resource consumption, but it turns out that I actually need to double my costs.

@aldy505
Copy link
Collaborator

aldy505 commented Jul 28, 2024

Thank you for this option. But how do requirements change exactly for errors-only mode? Previously, 8 GB RAM was enough for the full set of features in version 23.7.0 (that I currently use in production), it was increased to 16 GB in version 23.11.1, and it looks like the same 16 GB is still required for the limited errors-only version 24.7.1.

I just wanted to get rid of all the features that I never use and be able to limit resource consumption, but it turns out that I actually need to double my costs.

To be honest, I don't think either of the employee or the community have measured it. We'll make an update when we do. It should be lower than the current spec though!

@dosinabox
Copy link

To be honest, I don't think either of the employee or the community have measured it.

For me, on Ubuntu 22.04.4 it's ~9-10 GB for errors-only vs ~10-11 GB for feature-complete. It's a bummer that the errors-only mode is not lightweight enough to be under 8 GB. I hope to see more improvements to this feature in the future, but until then I'll stick to the older versions.

@github-actions github-actions bot locked and limited conversation to collaborators Aug 13, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

6 participants