Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server: B should not retry transcoding if client closed/canceled the connection #2124

Merged
merged 7 commits into from
Dec 6, 2021

Conversation

leszko
Copy link
Contributor

@leszko leszko commented Dec 2, 2021

What does this pull request do? Explain your changes. (required)
B will take any more attempts of sending the segment to O if the client closed/canceled the request connection.

Before, even if the client was not interested in transcoding anymore, B tried transcoding until the maxAttemps number is reached.

Specific updates (required)

  • Use the request context to determine if B should retry the transcoding

How did you test each of these updates (required)

Introduced println and sent requests from the script for two options:

  • request is suddenly closed => observed no retries
  • request is kept open => retries occurred

Does this pull request close any open issues?
fix #2113

Checklist:

Copy link
Member

@victorges victorges left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

server/broadcast.go Outdated Show resolved Hide resolved
assert.NotNil(err)
assert.Equal("Hit max transcode attempts: UnknownResponse", err.Error())
assert.Contains("context canceled", err.Error())
assert.Equal(2, transcodeCalls, "Segment submission calls did not match")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be 1?

Better yet, should we check if the context is cancelled before we make the attempt, so that we don't try even once in case the context is cancelled before we start trying?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it shouldn't be one. This value is aggregated across the whole test. Meaning:

  1. +1 in line 781
  2. +1 in line 791 (because we try one time to transcode)

Is it good or bad that we have one unit test for basically one function, not a separate function for one test scenario, it's a good question. I'd personally change it to have one unit test function per each scenario, but here I want to be consistent with what we do in the whole project.

Better yet, should we check if the context is cancelled before we make the attempt, so that we don't try even once in case the context is cancelled before we start trying?

It boils down to what we discussed #2124, especially these comments:

#2113 (comment)

It would make sense to me to always make a "reasonable attempt" to complete the transcode if a segment was (fully?) received, regardless of client connection status.

And

#2113 (comment)

it doesn't completely break the ability to use ffmpeg to push segments and then query for a playlist after

The thing is that if we don't try to transcode even once, then If the connection is suddenly closed, we don't have any results.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thing is that if we don't try to transcode even once, then If the connection is suddenly closed, we don't have any results.

Makes total sense, agreed!

No, it shouldn't be one. This value is aggregated across the whole test. Meaning:

I see... Yeah, this running of multiple test cases on the same test is quite confusing right now, so IMO it's bad. We could theorically create some setup functions and call them before each case, but there are some nice test libraries out there that we could use which make this much easier.

One of those that I really like (but also one of the few that I have XP with) is goconvey which has some nice sugar for setting things up so that you write the setup code on the same function where you're going to use it, but it's also re-executed for each "leaf" test function (parent functions basically work as "before each" steps). An example of where I've used it: https://github.com/victorges/nuledger/blob/main/authorizer/handler_test.go#L68

As for our current case, WDYT of resetting the transcodeCalls on the beginning of each test case, so that we can deal with a more reasonable number in each of these cases? Could be done in the same line where we set the MaxAttempts like MaxAttempts, transcodeCalls = 10, 0

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either way, total optional/nitpick change now that I understood this is correct. Feel free to just resolve the comment and leave this as is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of those that I really like (but also one of the few that I have XP with) is goconvey which has some nice sugar for setting things up so that you write the setup code on the same function where you're going to use it, but it's also re-executed for each "leaf" test function (parent functions basically work as "before each" steps). An example of where I've used it: https://github.com/victorges/nuledger/blob/main/authorizer/handler_test.go#L68

Yeah, In my opinion, it'd also be better. Now these tests are messy.

As for our current case, WDYT of resetting the transcodeCalls on the beginning of each test case, so that we can deal with a more reasonable number in each of these cases? Could be done in the same line where we set the MaxAttempts like MaxAttempts, transcodeCalls = 10, 0

Yeah, good idea. Updated.

Copy link
Contributor Author

@leszko leszko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review @victorges

server/broadcast.go Outdated Show resolved Hide resolved
assert.NotNil(err)
assert.Equal("Hit max transcode attempts: UnknownResponse", err.Error())
assert.Contains("context canceled", err.Error())
assert.Equal(2, transcodeCalls, "Segment submission calls did not match")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it shouldn't be one. This value is aggregated across the whole test. Meaning:

  1. +1 in line 781
  2. +1 in line 791 (because we try one time to transcode)

Is it good or bad that we have one unit test for basically one function, not a separate function for one test scenario, it's a good question. I'd personally change it to have one unit test function per each scenario, but here I want to be consistent with what we do in the whole project.

Better yet, should we check if the context is cancelled before we make the attempt, so that we don't try even once in case the context is cancelled before we start trying?

It boils down to what we discussed #2124, especially these comments:

#2113 (comment)

It would make sense to me to always make a "reasonable attempt" to complete the transcode if a segment was (fully?) received, regardless of client connection status.

And

#2113 (comment)

it doesn't completely break the ability to use ffmpeg to push segments and then query for a playlist after

The thing is that if we don't try to transcode even once, then If the connection is suddenly closed, we don't have any results.

Copy link
Contributor

@darkdarkdragon darkdarkdragon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@leszko leszko merged commit e698fd0 into livepeer:master Dec 6, 2021
@leszko leszko deleted the 2113-b-cancel-segment branch December 6, 2021 09:10
leszko added a commit to leszko/go-livepeer that referenced this pull request Dec 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

B should cancel segment processing if the HTTP push client request ends
3 participants