Dynamically change log level by sending a signal

[GiorgioRegni]

It would be very useful on production system to be able to change log level to debug when receiving a SIGUSR1 signal and reverting to the configured log level when receiving that signal again.
Maybe SIGUSR2 to go intro trace mode as well.

This way we could dynamically switch between different log levels on a process without restarting it.

[ghost]

Well, I'm not against setting that in werelogs, but does it not have its place in the code using werelogs instead ? Not that werelogs is used by a lot of tools, but I feel it would be more appropriate to handle signals not in a lib, but instead in the code of a service itself.

Still, I'm guessing this requests means to include that in a library ?

[msegura]

isn't the place of library to include code that would avoid duplicating the same code everywhere else? I am lost.

[ghost]

I'd say that it's slightly different. A library's aim is to provide a re-usable feature, that can be self-contained, and not interfere with the code built around the said feature. Note that I understand why you may see that as a code duplication, but we also need to take into account the notion of concerns separation.

Playing with signals within a library means:

1. We are imposing the decision of tweaking signals to tweak the log configuration, despite what any user may think (remember, this library is open-source, and not purely internal, so there can be other users than just @scality)
2. We are interfering with whatever signal setup can be setup by a program including this library (to get the logging features).

This is why I'm saying that if we want to do that, I personally think that it'd be better to do this in our S3, MD, and Vault repositories, ie. the code that is responsible for the behavior of the process as a whole. They are responsile for the management of their process's behavior, and as such, should be owner of the tuning over those.

[ThibaultRiviere]

Agree with @DavidPineauScality !

I don't know for you guys but signal is maybe not the best solution, why not go through the server of the services .

• sh curl -d '{ "logLevel": "debug", "dumpLevel": "error" }' ip:port/config/logger

[msegura]

@ThibaultRiviere @DavidPineauScality the curl looks great. Could we look into implementing that?

[ghost]

That confirms that we need to avoid doing that in werelogs, and instead, do that in S3, Vault, Metadata (my point from the beginning).

Only question: DO we add a HTTP server in the Repds for this, @ThibaultRiviere ?

[ThibaultRiviere]

@DavidPineauScality No http on the repd side .
But we probably can go through bucketd for transfer the request and change the repd log level.

[ghost]

Now this gets me wondering: What if we wanna change the log level because some kind of weird state kciks in, where the component may not answer some network-based requests ? This is one case in favor of the signal-based management (still in S3, Md, Vault, not here), against curl-based.

I don't have any strong opinion on this specific detail, as long as it's done in the right repos.

[msegura]

@DavidPineauScality the signal approach loooks great as well !