[1.9.0] Interlaced video dropping frames

[v2k]

We're having issues with interlaced signals. Primarily we've been testing 1080i59.94. Progressive signals appear to work without issue.

v1.8.6 with 11+ Blackmagic drivers (we've tried up to latest 12.1)
Windows
Unity 2019.4.28f1
1 Sub Device Full Duplex
DeckLink 8k Pro via SDI

We've noticed frames dropping. Narrowing it down, we're testing even outside of our application and with your 07_InputModifyOutputDemo sample, we still see frames dropping. We've tried this on various systems with ranging specs, but nothing has appeared to run without dropping frames.

[AndrewRH]

Please could you try the new 1.9.0 version - we spent a lot of time improving performance and addressing frame dropping.

Let me know how that goes for you.

Thanks,

[drinsley]

I made a build with the 07_InputModifyOutputDemo 1.9.0 version
Input is 1080i5994, 10bit 422 YUV
Set it to output 1080i5994, 8bit 4444 ARGB

It's hitching every 10 seconds or so.
12.1 BM drivers
Unity 2019.4.28f1
1 Sub Device Full Duplex
DeckLink 8k Pro via SDI

[AndrewRH]

Could you check the profiler? Perhaps it's due to garbage collection?
Have you tried running a build outside of Unity editor?

Thanks,

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[RichRH]

Hi @v2k - it's been a while, did you manage to try v1.9.0 and see any better results?

Hi @drinsley - just checking if this is still an issue for you? If so, when you're seeing the hitching is this on the final output over SDI and does it correspond with a dropped frame in the 'DeckLinkOutput' -> 'Preview' component?

I've run a few tests here and haven't see any obvious dropping but would love to try and track down any outstanding glitching..

Cheers,

[v2k]

@RenderHeadsMrT we had to abandon the plug-in, could not get it to work smoothly with interlaced source. I believe we tried the updated version to no avail.

[RichRH]

Sorry to hear you had to abandon it @v2k - my interpretation from your inital report is interlaced input could be the root of this, which hasn't had the same performance pass as output. We hope to address this soon and may hopefully see you again in the future.

Cheers,

[drinsley]

I am running version 1.6.6 and comparing it to v1.9.3 and the v1.6.6 plays the interlaced much more smoothly than the latest version. Is version v1.6.6 playing better because it sees the video as 8-bit instead of 10bit?

Video input is 1080i5994 10-bit 4:2:2 YUV.

AVPro DeckLink 1.6.6
Blackmagic v12.4 drivers installed, it says its using Decklink API version 10.11.4.0
Windows 10 Home 64
Unity 2019.4.40f1
Changes device to 1080i59.94 8-bit 4:2:2 YUV
DeckLink 8k Pro via SDI

AVPro DeckLink 1.9.3
Blackmagic v12.4 drivers installed, it says its using Decklink API version 12.2.0.0
Windows 10 Home 64
Unity 2019.4.40f1
Changes device to 1080i59.94 10-bit 4:2:2 YUV
DeckLink 8k Pro via SDI

[RichRH]

Thanks for the update @drinsley - 8-bit/10-bit could be a clue but I did also find a bug in some recent rendering code that could have impacted frame capture. I will try and replicate as close to your setup as possible and see if I can track down any further issues.

Cheers,

[v2k]

Any luck with reproducing the issue?

[RichRH]

Hi @v2k (and @drinsley),

I've been looking into the interlaced output side of things (#60 - first so that I could get loopback working) and located/fixed an issue there that I believe could be similar to the above. The input side is slightly trickier to resolve but I'm hoping to have it in a new release by early next week.

Cheers,

[drinsley]

That's great, we'll test it when it comes out

[RichRH]

Hi @v2k and @drinsley,

1.9.4 just released which hopefully fixes the above for you - will close this ticket but please do reopen if you're still seeing issues.

Cheers,

[drinsley]

So we tested it, and it runs well, but there is ghosting on output
If you look at the input video and compare it to the output video. When there is a fast pan or a player running increases their speed. You can see that there is a frame behind him and it isn't synced.

Also in our application that we're using within Unity it's showing that we're dropping frames every 10 seconds or so. The 07_InputModifyOutputDemo scene does not seem to drop. But they both have the ghosting on output.

Same setup as before using Shogun Atomos for input and viewing output.
AVPro DeckLink 1.9.4
Blackmagic v12.4 drivers
Windows 10 Home 64
Unity 2019.4.40f1
1080i59.94 10-bit 4:2:2 YUV input
DeckLink 8k Pro via SDI

[Chris-RH]

Hi @drinsley,
Would you be able to provide a video demonstrating this behaviour please?

[drinsley]

Ok I sent the videos along with a image showing the difference with a jpeg.

I can't reliably reproduce the dropping on our other system so I didn't send a video of that.

[Chris-RH]

Thanks for the email. Our dev will look into it.

[v2k]

Any updates on this?

[Chris-RH]

No update as of yet. We are a very small team, so tend to focus on resolving a few issues for one asset at a time.

[RichRH]

Hi @drinsley / @v2k,

After some digging, I believe I've located the interlaced ghosting issue - whilst I look into potential code solutions, you might be able to try the following:

• Fundamentally, I believe it's down to script execution order - in our '07_InputModifyOutputDemo' scene, input is understandably scheduled before output, but the issue here is interlaced output is split across two frames (odd/even) and as output follows input, the first input frame doesn't get a chance to be rendered to camera before being grabbed by the first output frame, but then a render does occur before the second frame, hence effectively outputting an amalgamation of two (full) input frames. The solution here seems to be swapping order, so output is scheduled first, effectively changing it to input -> render -> output (from input -> output -> render): https://docs.unity3d.com/Manual/class-MonoManager.html
• I tested the above in conjuction with setting our 'DeckLinkOutput' component to be 'Synced To Input' (this has code to sync alternating output frames against a full input frame), and also changing our 'DeckLinkManager' -> 'Deinterlace Method' to 'None' (to avoid interlacing blended input frames)

I hope that helps.

Cheers,

[Chris-RH]

@drinsley @v2k
Has this helped at all?

[Chris-RH]

This has been fixed in the current release, please let us know if this is still a problem for you.

[drinsley]

What are your settings for the latest version in 1.9.5?
Are you running it in 2022?
We are currently trying to compare the multiple versions

[RichRH]

Hi @drinsley,

Thanks for confirming you're looking into this.

We've mostly been testing with 2021, using the videos you sent via email. We feed the original video over a 1080i59.94 10-bit 4:2:2 YUV signal (SDI) into our '07_InputModifyOutputDemo' scene then output using the same signal on loopback into another input and capture that using Blackmagic Media Express. Our Unity components are configured as follows:

• Input: Auto
• Output: 1080i59.94 10-bit 4:2:2 YUV and 'Synced To Input' (using the only input in the scene)
• Interlacing set to 'None' ('DeckLinkManager' -> 'Deinterlace Method')

We then compare the captured video to your comparison jpeg, and with the latest changes are seeing a match to the original (left), rather than ghosted (right) version.

Cheers,

[v2k]

We made some progress on this issue. The script order was an issues - especially aligning with our render pipeline.

We still found some hitching with this solved, but we haven't had time to investigate further.

[Chris-RH]

Great. Please let us know if there is anything that needs to be in our documentation that might help other users :)