Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core: reuse remote transcoder for stream #1849

Merged

Conversation

reubenr0d
Copy link
Contributor

@reubenr0d reubenr0d commented Apr 21, 2021

What does this pull request do? Explain your changes. (required)

Specific updates (required)

  • Adds map to keep track of transcoders assigned to sessions
  • Updated selectTranscoder to return and set transcoders to sessions
  • Removed transcoder from map in case of session end/transcoder errors

How did you test each of these updates (required)

On local offchain setup:

  • Setup 1 O, 2 T, 1 B
  • Start stream and observed that only one transcoder is processing stream
  • Killed that transcoder and stream moves to other transcoder
  • Created Unit Tests

Does this pull request close any open issues?

Fixes #1842
Fixes #1712
Fixes #1273

Checklist:

core/orchestrator.go Outdated Show resolved Hide resolved
core/orchestrator.go Outdated Show resolved Hide resolved
core/orchestrator.go Outdated Show resolved Hide resolved
@kyriediculous
Copy link
Contributor

We also want to remove the currentTranscoder from the streamSessions map when it returns an error here

res, err := currentTranscoder.Transcode(md)

@kyriediculous
Copy link
Contributor

We also want to remove the currentTranscoder from the streamSessions map when the connection closes like we do here

delete(rtm.liveTranscoders, transcoder.stream)

@reubenr0d reubenr0d force-pushed the reuse-transcoder-for-stream branch 4 times, most recently from 5b8003e to cb99f0c Compare April 25, 2021 16:03
core/orchestrator.go Outdated Show resolved Hide resolved
core/core_test.go Outdated Show resolved Hide resolved
core/orch_test.go Outdated Show resolved Hide resolved
core/orch_test.go Outdated Show resolved Hide resolved
core/orch_test.go Outdated Show resolved Hide resolved
core/orchestrator.go Show resolved Hide resolved
core/orchestrator.go Outdated Show resolved Hide resolved
@reubenr0d reubenr0d force-pushed the reuse-transcoder-for-stream branch 4 times, most recently from 358d391 to b248bef Compare April 25, 2021 17:18
@reubenr0d reubenr0d marked this pull request as ready for review April 25, 2021 17:24
core/orch_test.go Outdated Show resolved Hide resolved
core/orch_test.go Outdated Show resolved Hide resolved
core/orch_test.go Outdated Show resolved Hide resolved
core/orch_test.go Outdated Show resolved Hide resolved
@reubenr0d reubenr0d force-pushed the reuse-transcoder-for-stream branch 2 times, most recently from 021aa53 to f659900 Compare April 28, 2021 07:56
Copy link
Contributor

@kyriediculous kyriediculous left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good ! Just two very small comments.

Feel free to rebase and flatten the commit history once you've addressed those.

Let's use a commit message that indicates which packages were altered and also use that in the title of the PR

e.g. "core: re-use remote transcoder for stream"

core/orch_test.go Show resolved Hide resolved
core/orch_test.go Outdated Show resolved Hide resolved
@kyriediculous kyriediculous changed the title Reuse transcoder for stream (#1842) core: reuse remote transcoder for stream Apr 28, 2021
CHANGELOG_PENDING.md Outdated Show resolved Hide resolved
core/orchestrator.go Outdated Show resolved Hide resolved
core/orchestrator.go Outdated Show resolved Hide resolved
core/orchestrator.go Show resolved Hide resolved
core/orchestrator.go Outdated Show resolved Hide resolved
@reubenr0d reubenr0d force-pushed the reuse-transcoder-for-stream branch 3 times, most recently from 3e8dcea to 59b545b Compare May 1, 2021 06:53
@reubenr0d
Copy link
Contributor Author

reubenr0d commented May 1, 2021

I've pushed the changes to the same commit(will use !fixup henceforth), have a look. The Ts are getting removed from the list properly now, but don't seem to adding back, if there is no error. This is the assertion in question. Shouldn't it be 2 after the T succeeds? I'm not quite sure how would it get selected, if the T was not in the list in the first place. Any advice?

@kyriediculous
Copy link
Contributor

I've pushed the changes to the same commit(will use !fixup henceforth), have a look. The Ts are getting removed from the list properly now, but don't seem to adding back, if there is no error. This is the assertion in question. Shouldn't it be 2 after the T succeeds? I'm not quite sure how would it get selected, if the T was not in the list in the first place. Any advice?

@reubenr0d Transcoder 2 gets removed here: https://github.com/reubenr0d/go-livepeer/blob/59b545b57a94f350879307da304a838e99d2c390/core/orch_test.go#L343

@reubenr0d
Copy link
Contributor Author

Transcoder 2 gets removed here

Right, the comment "assert transcoder gets added back to remoteTranscoders if no transcoding error" threw me off, if it's not supposed to get added back, shall I just remove the last set of assertions after that comment?

core/orchestrator.go Show resolved Hide resolved
core/orchestrator.go Show resolved Hide resolved
@reubenr0d reubenr0d force-pushed the reuse-transcoder-for-stream branch from f45fb1b to e5654d8 Compare May 4, 2021 13:36
@kyriediculous
Copy link
Contributor

@yondonfu Did you have any other remarks regarding the changes you requested ?

@yondonfu
Copy link
Member

yondonfu commented May 4, 2021

See #1864 (comment)

Copy link
Member

@yondonfu yondonfu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@kyriediculous kyriediculous merged commit 62b4830 into livepeer:master May 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants