Unity Audio delay and crackling when using multiple audio inputs

[Kaimen001]

Describe the bug
Hi, I have a DeckLink Quad HDMI recorder. I am capturing video and audio through HDMI. I have verified that the audio and video are synchronized in Media Express but in the application in unity the audio comes out with a considerable delay. Initially of several seconds but if I turn the camera off and on the delay stays in less than a second but very visible. I have tried the 01_InputDemo scene and the same thing happens, a delay of less than a second.

Your Setup (please complete the following information):

• Unity version: 2020.3.18f1
• AVPro DeckLink version: 1.9.1
• Operating system version: Windows 10
• Hardware model: DeckLink Quad HDMI
• Cables used: HDMI
• Desktop Video version: 12.2
• Video mode (resolution, frame-rate, pixel-format): 1920x1080 50fps, 10‑bit 4:2:2

To Reproduce
Steps to reproduce the behavior:

1. Open 01_InputDemo scene
2. I compile the scene and the audio delay can be seen

[Kaimen001]

Hello,
I have tried on another computer with a Blackmagic DeckLink 8K Pro, w10, NVIDIA RTX A4000 and I have the same problem. It is an audio delay of a second or less but very annoying.
I have made a new project in unity 2020.3.18f1, I have imported AVPro DeckLink, I have compiled the scene "01_InputDemo" and I have this delay. Could there be something in the project setup that I need to consider? I need to solve this problem. Thanks a lot.

[AndrewRH]

Hi

Have you tried Audio Settings (in Project Settings) changing the audio buffer size?

Thanks,

[Kaimen001]

Hello AndrewRH,
I have been playing with audio setting. With both best latency and good latency, there are improvements in synchronization between audio and video, although it is not perfect. When I use one capture channel from the DeckLink Quad HDMI everything seems to be fine but if I capture two or more sources the audio lags and even breaks up. I've tried increasing the buffer size but it doesn't improve the delay. I don't know if it's a limitation of the unity audio system but it doesn't work well. What audio setup do you recommend? Do you know if fmod or another audio system can solve the problem? I can't find anything that can give me an idea how to fix this.
I look forward to your help. Thanks a lot.

[Kaimen001]

Hello again AndrewRH,
Actually the problem that worries me the most is when I capture two or more videos with audio at the same time. If I only capture the audio for one of the inputs and disable the audio for the others, everything seems fine. But when I try to capture video and audio from more than one input, the capture audio crackles a lot and slows down more and more. I have tried various configurations, changing the capture inputs but the result is always the same. I also don't have many audio configuration options. Any ideas.
Thanks a lot.

[kahnivore]

Are you running with vsync on or off? And which Decklink hardware are you using?

Also, would you be able to share an audio clip of the crackling?

[Kaimen001]

Hi AndrewRH,
gresy with vsync turned off. I'm using DeckLink Quad HDMI and a Quadro A6000 graphics card and a Quadro SYNC II. I tried it on another computer (I don't have it anymore) with Blackmagic DeckLink 8K Pro and a Quadro A4000 (without Quadro SYNC II) and the problem is the same. I attach a video with the audio crackles and, although you can't hear it, it goes very slowly and with more and more delay.
Thanks a lot

VideoAudioCrackles_1.mp4

.

[Kaimen001]

Hello, have you been able to see something?
Thanks

[AndrewRH]

This seems like it could be performance related. Perhaps the C# code for grabbing the audio is not very optimal and required an optimisation pass. Poor performance could lead to crackling and added delays due to buffers not being consumed fast enough. It also sounds like a potential issue with the low-level code for multiple inputs. This needs further investigation.

[AndrewRH]

Could be related to multiple input audio issues mentioned in #47

[pilzinho]

We have the same audio issues with the same Decklink hardware. When using audio buffer size for best latency and only capture one video+audio input it's kind of ok. But also already a visible audio lag and it goes out of sync pretty easily if the FPS drop at some point. Audio dropouts are also very common and not very pleasant. Capturing a second input with audio at the same time only works on rare occasions. Although it is probably not necessary for us to capture multiple audio streams at the same time it is an absolute must for our clients to be able to capture all video inputs and enable audio of one of the inputs at any point.

[RichRH]

FYI.. I've tracked down and fixed an issue causing massive audio thread stalls (especially with multiple inputs, hence audio playback breaking up). Currently looking into seeing if anything can be done to mitigate the delay.

Cheers,

[Chris-RH]

This issue has been fixed in the latest release (AVPro Decklink version 1.9.7). Please let us know if this issue has not been fixed for you.