Ensure signature comparison success ratio among Nvidia and CPU signatures

[cyberj0g]

livepeer/go-livepeer#2654

Changes

• Add signature files calculated during Nvidia and CPU transcoding
• Add a unit test, which calculates and ensures that false negative (signature mismatch for matching renditions) ratio is less than 1% between Nvidia and CPU signatures
• Add a unit test, which calculates and ensures that false positive (signature match for different renditions) ratio is less than 15% for both CPU and Nvidia signatures

For this unit test to pass, a PR which sets threshold to detect frames as similar in signature comparison function of Ffmpeg needs to be merged first.

Below are results for the threshold value of 116 (default) and 150 (new value). I think, such an increase in FP ratio is acceptable. New value was picked by hand in a few attempts, to reduce the chance of overfitting our limited test dataset.

th_xh (thl1) = 116
----------------
CPU 360p False Positive rate: 0.091133
CPU 720p False Positive rate: 0.110837
Nvidia 360p False Positive rate: 0.091133
Nvidia 720p False Positive rate: 0.110837
CPU - Nvidia 360p False Negative rate: 0.068966
CPU - Nvidia 720p False Negative rate: 0.000000

th_xh (thl1) = 150
----------------
CPU 360p False Positive rate: 0.103448
CPU 720p False Positive rate: 0.125616
Nvidia 360p False Positive rate: 0.113300
Nvidia 720p False Positive rate: 0.128079
CPU - Nvidia 360p False Negative rate: 0.000000
CPU - Nvidia 720p False Negative rate: 0.000000

[yondonfu]

Curious if generating signatures instead of checking them in would be possible since that would allow us to do comparisons of signatures generated using the current ffmpeg impl being used? Would also reduce repo clutter. Thoughts?

[cyberj0g]

Thanks @yondonfu, all comments are addressed. We can generate signatures right in the test, since lpms tests are running on GPU runner now.

[yondonfu]

LGTM!

[yondonfu]

This setup is much cleaner!