server/broadcast: don't retry segments after max attempts

[jailuthra]

What does this pull request do? Explain your changes. (required)

Return 422 to push-api clients once the B has retried a segment MaxAttempts times.

Specific updates (required)

See commit history

How did you test each of these updates (required)

1. Unit test to ensure 422 is returned when retried a segment MaxAttempts times
2. Manual testing of Push API after returning an error for transcoding:

diff --git a/server/broadcast.go b/server/broadcast.go
index ff17a0d5..34c89e37 100644
--- a/server/broadcast.go
+++ b/server/broadcast.go
@@ -612,6 +612,9 @@ func transcodeSegment(cxn *rtmpConnection, seg *stream.HLSSegment, name string,
                        err = errors.New("empty response")
                }
                return nil, info, err
+       } else if sess != nil {
+               cxn.sessManager.completeSession(sess)
+               return nil, info, errors.New("Unknown error")
        }

        // download transcoded segments from the transcoder

Does this pull request close any open issues?

Fixes #2069

Checklist:

• Read the contribution guide
• make runs successfully
• All tests in ./test.sh pass
• README and other documentation updated
• Pending changelog updated

[yondonfu]

LGTM

[yondonfu]

I was thinking about whether there could be a case where B runs out of sessions before it hits the max # of attempts. Dumping some thoughts below just for visibility.

Based on

go-livepeer/server/broadcast.go

Line 775 in fbcdccb

for len(attempts) < MaxAttempts {

I believe that if B runs out of sessions, it will still hit the max # of attempts - each of the remaining attempts will just result in an error caused by B being out of sessions. And since we check for a non-retryable error here and return before the no sessions available error check in the HTTP push handler we will always return a 422 in this scenario before getting to the code that triggers a 503.