Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split devtool Orchestrator scripts with/without external transcoder. Prevent run scripts from using same CLI port. #2299

Merged
merged 2 commits into from
Mar 10, 2022

Conversation

thomshutt
Copy link
Contributor

@thomshutt thomshutt commented Mar 7, 2022

What does this pull request do? Explain your changes. (required)
It's currently not possible to run the run_broadcaster and run_transcoder scripts generated by devtool simultaneously, because run_broadcaster specifies a CLI Port of 7935 and run_transcoder doesn't specify one (and so go-livepeer uses 7935 by default).

Specific updates (required)

  • Specify <orchestrator port + 1> as the CLI port for the transcoder script
  • Split out Orchestrator script into two separate ones (with/without transcoder running)
  • Update docs to reflect script changes

How did you test each of these updates (required)

  • Generated the scripts and streamed with both B/O and B/O/T setup

Does this pull request close any open issues?
No

Checklist:

Copy link
Contributor

@leszko leszko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thomshutt Thanks for the PR.

I wonder if this change actually solves the issue of executing run_transcoder.sh. Here are my thoughts:

  1. The O/T topology (when the orchestrator is run together with transcoder in one process) works anyway, just execute:
./run_broadcaster*.sh
./run_orchestrator*.sh

There is no port conflict, everything works fine.

  1. The split OT topology (which your PR is supposed to fix) didn't work for me after your change. Maybe I did something wrong...

I think to make the split OT topology it'd require some more changes. Even if we fix the CLI port, the following execution does not work.

./run_broadcaster*.sh
./run_orchestrator*.sh
./run_transcoder*.sh

The reason for this is that the Orchestrator is anyway run with -transcoder parameter, so it does not accept other transcoders.

My suggestion is to:

  • either fix scripts for the split O/T topology
  • either just accept the split O/T does not work and maybe create a GH issue to fix it

@thomshutt
Copy link
Contributor Author

@leszko Thanks for reviewing!

That makes sense, I'll remove the -transcoder parameter, since all of the documentation (and in particular https://github.com/livepeer/go-livepeer/blob/master/cmd/devtool/README.md) talks about B/O/T as separate entities and having a Transcoder automatically started by the Orchestrator confused me when I was getting this set up.

@leszko
Copy link
Contributor

leszko commented Mar 7, 2022

TBH I'd keep the possibility for the O/T topology and maybe add an option to generate the split O/T (separate transcoder)? I think O/T is good, because it's the simplest way to test something on-chain. So I use it very often.

@thomshutt
Copy link
Contributor Author

👍 Fair enough, I'll go with that then

@thomshutt thomshutt requested a review from leszko March 7, 2022 15:26
Copy link
Contributor

@leszko leszko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the change. I like it. And also great that you added run*.sh to .gitignore. Added some comments.

CHANGELOG_PENDING.md Outdated Show resolved Hide resolved
cmd/devtool/README.md Outdated Show resolved Hide resolved
cmd/devtool/devtool.go Show resolved Hide resolved
cmd/devtool/devtool.go Show resolved Hide resolved
@@ -330,13 +330,10 @@ func ethSetup(ethAcctAddr, keystoreDir string, isBroadcaster bool) {
func createTranscoderRunScript(ethAcctAddr, dataDir, serviceHost string) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not related to your PR, but we could clean up a little bit the generated scripts, now some of the commands are in new lines, some of them are in the same, I'd make it consistent. Wdyt?

Generated run_broadcaster.sh (some parameters are in new lines, some are in the same):

#!/bin/bash
./livepeer -v 99 -ethController 0x77a0865438f2efd65667362d4a8937537ca7a5ef -datadir ./.lpdev2/broadcaster_0x06AAaf48472D6a64f33C7e281b5dB0fE419878F6 \
    -ethAcctAddr 0x06AAaf48472D6a64f33C7e281b5dB0fE419878F6 \
    -ethUrl http://localhost:8545/ \
    -ethPassword "" \
    -network=devenv \
    -blockPollingInterval 1 \
    -monitor=false -currentManifest=true -cliAddr 127.0.0.1:7935 -httpAddr 127.0.0.1:8935  -broadcaster=true -rtmpAddr 127.0.0.1:1935

Generated run_trascoder.sh (everything is in one line):

#!/bin/bash
./livepeer -v 99 -datadir ./.lpdev2/transcoder_0x065195849B39Bf5D99b4EbD28F61C75C6EB8771a -orchSecret secre -orchAddr 127.0.0.1:8936 -cliAddr 127.0.0.1:7937 -transcoder

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thomshutt Could you check this as well?

Copy link
Contributor

@leszko leszko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Before merging please:

  1. Look also into this comment.

  2. Squash, rename, and rebase your commits according to what's described in the Contribution Guide: Commits

cmd/devtool/devtool.go Show resolved Hide resolved
@@ -330,13 +330,10 @@ func ethSetup(ethAcctAddr, keystoreDir string, isBroadcaster bool) {
func createTranscoderRunScript(ethAcctAddr, dataDir, serviceHost string) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thomshutt Could you check this as well?

@thomshutt thomshutt changed the title Prevent Transcoder/Broadcaster run scripts from using same CLI port Split devtool Orchestrator run scripts into versions with/without external transcoder and prevent Transcoder/Broadcaster run scripts from using same CLI port Mar 10, 2022
@thomshutt thomshutt changed the title Split devtool Orchestrator run scripts into versions with/without external transcoder and prevent Transcoder/Broadcaster run scripts from using same CLI port Split devtool Orchestrator scripts with/without external transcoder. Prevent run scripts from using same CLI port. Mar 10, 2022
@thomshutt thomshutt merged commit 78f59c9 into master Mar 10, 2022
@thomshutt thomshutt deleted the script-port-conflict branch March 10, 2022 12:07
@thomshutt
Copy link
Contributor Author

Thanks for all your help here @leszko 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants