FireFox publisher removes a video track, janus sends changing simulcast layers on the subscriber side

[JanFellner]

What version of Janus is this happening on?
#c684d43d

Have you tested a more recent version of Janus too?
there is none

Was this working before?
Did not see this behaviour in the non multistreaming version

Is there a gdb or libasan trace of the issue?
Not dump but signalling related

Additional context
I have a publisher and a subscriber in the videoroom.
The publisher is using firefox.
The publisher is publishing audio and video and removes video by replaceTrack(null) on the sender in the peerConnection.
There is no SDP exchange involved as the peerConnection handles this without a negotiation request.

Janus then starts to send simulcast layer changes on the subscription each second. It never stops, just goes on and on.
I did not see this if the publisher is using chrome as browser.

Each second janus sends two events.
{
"": "2022-03-18T13:55:36.365Z - ws - in",
"janus": "event",
"plugindata": {
"data": {
"mid": "1",
"room": "R_4209a3ab63ac4db9a4a3756b15021db3",
"substream": 0,
"videoroom": "event"
},
"plugin": "janus.plugin.videoroom"
},
"sender": 5799355206759672,
"session_id": 4856516226201193
}
{
"": "2022-03-18T13:55:36.376Z - ws - in",
"janus": "event",
"plugindata": {
"data": {
"mid": "1",
"room": "R_4209a3ab63ac4db9a4a3756b15021db3",
"substream": 2,
"videoroom": "event"
},
"plugin": "janus.plugin.videoroom"
},
"sender": 5799355206759672,
"session_id": 4856516226201193
}
{
"": "2022-03-18T13:55:37.387Z - ws - in",
"janus": "event",
"plugindata": {
"data": {
"mid": "1",
"room": "R_4209a3ab63ac4db9a4a3756b15021db3",
"substream": 1,
"videoroom": "event"
},
"plugin": "janus.plugin.videoroom"
},
"sender": 5799355206759672,
"session_id": 4856516226201193
}
{
"": "2022-03-18T13:55:37.397Z - ws - in",
"janus": "event",
"plugindata": {
"data": {
"mid": "1",
"room": "R_4209a3ab63ac4db9a4a3756b15021db3",
"substream": 2,
"videoroom": "event"
},
"plugin": "janus.plugin.videoroom"
},
"sender": 5799355206759672,
"session_id": 4856516226201193
}

[JanFellner]

In the janus console i see the request for a PLI....

[Fri Mar 18 14:01:04 2022] [WARN] [3478362136411000] Unknown SSRC, dropping packet (SSRC 2911694313)...
[Fri Mar 18 14:01:04 2022] [WARN] [3478362136411000] Unknown SSRC, dropping packet (SSRC 2911694313)...
[Fri Mar 18 14:01:04 2022] [WARN] No packet received on substream 2 for a while, falling back to 1
[Fri Mar 18 14:01:04 2022] We need a PLI for the simulcast context
[Fri Mar 18 14:01:04 2022] Simulcast change sending PLI to 491dae36-e798-440a-97f5-22793f1740b0 (#1, P_156b2da8f3d14586854bdd488e004150)
[Fri Mar 18 14:01:04 2022] Received keyframe on #1 (SSRC 2041953786), switching (was #2/3659139286)
[Fri Mar 18 14:01:04 2022] [5799355206759672] Sending event to transport...
[Fri Mar 18 14:01:04 2022] SSRC changed, 3659139286 --> 2041953786
[Fri Mar 18 14:01:04 2022] Computed offset for RTP timestamp: 87801
[Fri Mar 18 14:01:04 2022] Received keyframe on #2 (SSRC 3659139286), switching (was #1/2041953786)
[Fri Mar 18 14:01:04 2022] [5799355206759672] Sending event to transport...
[Fri Mar 18 14:01:04 2022] SSRC changed, 2041953786 --> 3659139286
[Fri Mar 18 14:01:04 2022] Computed offset for RTP timestamp: 27

In case i should provide more context (SDP, or larger logs, i am happy to do so)

[fippo]

and what happens on the network? You know the drill ;-)

[lminiero]

Please use quotes for snippets, since otherwise anything with a hash (e.g. #1) becomes an implicit link to an issue. I've edited the comment for you. That said, have you checked if this happens on 0.x too? I suspect this might be a consequence of the check we made to notify things right away, and/or a recent fix on simulcast layers being triggered only when 0 was stopped.

[lminiero]

and what happens on the network? You know the drill ;-)

That would help, as if the plugin is notifying substream changes it's because it's seeing media in for those SSRCs.

[JanFellner]

We are still heavily working on all the changes we need to do for multistreaming. We have a ui that allowed us to capture pcap traces from janus in the past 0.x branch and i will revive that one on monday/tuesday which then allows me to capture unencrypted pcap traces.

That would help, as if the plugin is notifying substream changes it's because it's seeing media in for those SSRCs.

Yeah that is what i did read from the Janus logs as well.

[lminiero]

I did a quick test to replicate, and with Wireshark I still do see video traffic coming in. The highest substream in particular (2) seems to have a keyframe sent once every second (I see a 90000 timestamp difference, and many packets with the same timestamp), which would explain why we switch to substream 2 (we see traffic for that substream, and it's what the user wants); it then consistently falls back to 1 because we have an automated fallback when no media for the current substream arrives for 250ms (which is true since Firefox is sending that video stream at 1fps).

As such, from what I can see I think Janus is doing what it should, and it sounds like a weird behaviour in Firefox when replaceTrack is called with null on a simulcast source.

[fippo]

tagging @jan-ivar - looks like reducing the framerate is not a good idea in this case?

[JanFellner]

I don´t expect ff to change or improve anything in webRTC...
So i solved it through additionally setting the transceiver to inactive and enforcing a SDP renegotiation with janus.
-> close ugly ff issue... :\