server: Fix timeout for stream recording background jobs #2573
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this pull request do? Explain your changes. (required)
We noticed a couple errors that happen when saving transcoded segments for livestream recordings.
These occur especially in
staging
for some reason, but the underlying cause can easily happen inproduction as well (and it often does).
In short terms, the issue is caused because saving the recorded segments is performed in a background
routine, while at the same time this routine uses the same
context.Context
to handle cancellation of thetask behind the scenes. This means that one the transcode request is finished (which is where the current
ctx
comes from, thehttp.Request
), the background saving operations will also be cancelled.This fixes it by enforcing a separate timeout on the background saves, independent of the request.
Specific updates (required)
How did you test each of these updates (required)
Automated tests. Pretty simple change as well, will check effect in staging on catalyst tests.
Does this pull request close any open issues?
livepeer/catalyst#126
Checklist:
make
runs successfully./test.sh
pass