Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: remote transcoders quietly getting dropped from selection #2707

Merged
merged 2 commits into from
Jan 17, 2023

Conversation

stronk-dev
Copy link
Contributor

@stronk-dev stronk-dev commented Jan 3, 2023

What does this pull request do?

Fixes a bug in split O/T setups, where a Transcoder can get dropped from selection alltogether, without any indication on the O or T side

See #2706 for more details

Specific updates

Seems to have been introduced in bf5397d so a looong time ago

How did you test each of these updates

Currently running the fix in production on the video miner pool. So far no connected transcoder has been dropped from the remoteTranscoders set as of yet

Does this pull request close any open issues?

Fixes #2706

@codecov
Copy link

codecov bot commented Jan 3, 2023

Codecov Report

Merging #2707 (bf9ec61) into master (a5606ca) will increase coverage by 0.01340%.
The diff coverage is 0.00000%.

❗ Current head bf9ec61 differs from pull request most recent head ab553b4. Consider uploading reports for the commit ab553b4 to get more accurate results

Impacted file tree graph

@@                 Coverage Diff                 @@
##              master       #2707         +/-   ##
===================================================
+ Coverage   56.33265%   56.34605%   +0.01340%     
===================================================
  Files             88          88                 
  Lines          19131       19130          -1     
===================================================
+ Hits           10777       10779          +2     
+ Misses          7765        7763          -2     
+ Partials         589         588          -1     
Impacted Files Coverage Δ
core/orchestrator.go 77.65517% <0.00000%> (ø)
server/mediaserver.go 67.56135% <0.00000%> (+0.20503%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a5606ca...ab553b4. Read the comment docs.

Impacted Files Coverage Δ
core/orchestrator.go 77.65517% <0.00000%> (ø)
server/mediaserver.go 67.56135% <0.00000%> (+0.20503%) ⬆️

@stronk-dev stronk-dev changed the title Md/remote transcoder fix Fix: remote transcoders quietly getting dropped from selection Jan 3, 2023
Copy link
Contributor

@cyberj0g cyberj0g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a nasty one, great finding! Before merging, could you please:

  • check why test of removeFromRemoteTranscoders doesn't catch that and update it
  • rebase cleanly on top of master

@stronk-dev
Copy link
Contributor Author

Two issues with the TestRemoveFromRemoteTranscoders:

  • At the point where it says it is removing the middle element, it is actually removing the final element, for which exists a separate code path
  • The bug only drops Transcoders with an index lower than i-1 and the size of the list when testing removal of a middle Transcoder is 3. Thus, when removing the Transcoder at index 1, the Transcoder at index 0 is still safe. The test would have to be done by removing a Transcoder at index 2 or higher where the index is not the last Transcoder in the list

I'll modify the test to reflect these changes

Copy link
Contributor

@cyberj0g cyberj0g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@cyberj0g cyberj0g merged commit 2a2b0b8 into master Jan 17, 2023
@cyberj0g cyberj0g deleted the md/RemoteTranscoderFix branch January 17, 2023 08:20
eliteprox pushed a commit to eliteprox/go-livepeer that referenced this pull request Feb 3, 2023
…livepeer#2707)

* Fix remote transcoders getting dropped from selection without error

* Fix TestRemoveFromRemoteTranscoders to correctly test removing transcoders from the middle of the set
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remote T's can get removed from the remoteTranscoders list if any T reconnects
2 participants