videoroom plugin does not set H264 profile-level-id in offer SDPs if room configuration doesn't have one set

[sdroege]

When publishing a H264 stream to the videoroom plugin, e.g. with the following SDP (offer snippet):

v=0
o=- 4347298905384029912 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
a=group:BUNDLE video0
m=video 9 UDP/TLS/RTP/SAVPF 96 97
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:UQdXcVwK+yoLMcQf1hEONcBrjFg269jF
a=ice-pwd:n19jeT55NWPYA1KMRhB6DBmK7uKi+Rcp
a=rtcp-mux
a=rtcp-rsize
a=sendonly
a=rtpmap:96 H264/90000
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 transport-cc
a=framerate:30
a=extmap:1 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=fmtp:96 packetization-mode=1;profile-level-id=42e01f;level-asymmetry-allowed=1
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=ssrc-group:FID 1663398171 960339433
a=ssrc:1663398171 msid:user1122501605@host-d269b0ed webrtctransceiver0
a=ssrc:1663398171 cname:user1122501605@host-d269b0ed
a=ssrc:960339433 msid:user1122501605@host-d269b0ed webrtctransceiver0
a=ssrc:960339433 cname:user1122501605@host-d269b0ed
a=mid:video0
a=fingerprint:sha-256 AD:70:24:06:67:60:CA:E0:D5:B3:6C:79:C0:1A:E8:A7:47:EF:01:02:93:8A:FD:6E:02:7F:82:6E:24:24:34:A5
a=rtcp-mux-only

the following answer is produced:

v=0
o=- 1628860567642678 1 IN IP4 157.90.220.103
s=VideoRoom 8444579664151715
t=0 0
a=group:BUNDLE video0
a=ice-options:trickle
a=fingerprint:sha-256 96:80:28:86:03:16:D1:EA:2E:5F:17:26:AB:5D:B4:21:5F:45:51:D8:5D:C4:54:D1:0C:61:88:5A:7E:A1:9A:32
a=msid-semantic: WMS janus
m=video 9 UDP/TLS/RTP/SAVPF 96 97
c=IN IP4 157.90.220.103
a=recvonly
a=mid:video0
a=rtcp-mux
a=ice-ufrag:f7wK
a=ice-pwd:5LkEOpHDUzOAIda8FSOvg9
a=ice-options:trickle
a=setup:active
a=rtpmap:96 H264/90000
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=extmap:1 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=fmtp:96 packetization-mode=1;profile-level-id=42e01f;level-asymmetry-allowed=1
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=msid:janus janusvideo0
a=ssrc:262240562 cname:janus
a=ssrc:3473806814 cname:janus
a=candidate:1 1 udp 2015363327 157.90.220.103 52070 typ host
a=end-of-candidates

Both contain the profile-level-id as expected. All good up to this point.

---

When subscribing to that stream, the following offer is created by the videoroom plugin:

v=0
o=- 1628860792898642 1 IN IP4 157.90.220.103
s=VideoRoom 8444579664151715
t=0 0
a=group:BUNDLE 0 1
a=ice-options:trickle
a=fingerprint:sha-256 96:80:28:86:03:16:D1:EA:2E:5F:17:26:AB:5D:B4:21:5F:45:51:D8:5D:C4:54:D1:0C:61:88:5A:7E:A1:9A:32
a=msid-semantic: WMS janus
m=audio 9 UDP/TLS/RTP/SAVPF 111
c=IN IP4 157.90.220.103
a=sendonly
a=mid:0
a=rtcp-mux
a=ice-ufrag:lrYr
a=ice-pwd:3CJkuRbS0C3bzgWwAc3pvp
a=ice-options:trickle
a=setup:actpass
a=rtpmap:111 opus/48000/2
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=msid:janus janus0
a=ssrc:3287662607 cname:janus
a=candidate:1 1 udp 2015363327 157.90.220.103 56412 typ host
a=end-of-candidates
m=video 9 UDP/TLS/RTP/SAVPF 107
c=IN IP4 157.90.220.103
a=sendonly
a=mid:1
a=rtcp-mux
a=ice-ufrag:lrYr
a=ice-pwd:3CJkuRbS0C3bzgWwAc3pvp
a=ice-options:trickle
a=setup:actpass
a=rtpmap:107 H264/90000
a=rtcp-fb:107 ccm fir
a=rtcp-fb:107 nack
a=rtcp-fb:107 nack pli
a=rtcp-fb:107 goog-remb
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:12 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=ssrc-group:FID 3262922593 4281109715
a=msid:janus janus1
a=ssrc:3262922593 cname:janus
a=ssrc:4281109715 cname:janus
a=candidate:1 1 udp 2015363327 157.90.220.103 56412 typ host
a=end-of-candidates

This does not include the profile-level-id anymore.

Firefox (88) does not seem to like that at all and produces an answer with the video stream being marked as inactive (and for some reason with VP8). With Chrome this works fine, and the same Firefox version handles H264 streams with profile-level-id just fine.

v=0
o=mozilla...THIS_IS_SDPARTA-88.0.1 8700073059593920207 0 IN IP4 0.0.0.0
s=-
t=0 0
a=fingerprint:sha-256 E2:28:38:4D:5D:EA:B7:C3:B1:E6:7D:CF:0C:36:44:74:7F:D5:34:49:36:96:42:8C:E7:87:14:33:6C:AB:EF:5F
a=group:BUNDLE 0
a=ice-options:trickle
a=msid-semantic:WMS *
m=audio 9 UDP/TLS/RTP/SAVPF 111
c=IN IP4 0.0.0.0
a=recvonly
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=fmtp:111 maxplaybackrate=48000;stereo=1;useinbandfec=1
a=ice-pwd:c04bcb10461b37ca8bec2e9c4110715b
a=ice-ufrag:a697b665
a=mid:0
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=setup:active
a=ssrc:3240949545 cname:{a46591d8-0996-4425-ba2d-f92ab7b5b769}
m=video 0 UDP/TLS/RTP/SAVPF 120
c=IN IP4 0.0.0.0
a=inactive
a=mid:1
a=rtpmap:120 VP8/90000

This (correctly) causes Janus to report an error: 465 Error processing SDP.

---

From my reading of the videoroom plugin, the profile-level-id is only ever put into the SDP if the configuration of the room has one set, which seems suboptimal.

IMHO Firefox is also too strict here, which is a bug on their side, but that the videoroom plugin is not forwarding this information (and probably other) from the publisher to the subscriber seems like not a good idea.

[tmatth]

@sdroege I think explicitly setting the profile-level-id for the room with the h264_profile field in e.g., the create request effectively hides this issue (EDIT: just saw that you already mentioned that at the end) but I think you're right, I wouldn't expect to have to set a profile like this if it's in the publisher SDP.

[sdroege]

Just to confirm, actually setting it in the room configuration causes it to appear in the offers.

[atoppi]

I can confirm the issue on the multistream branch.
With master it works as intended.

[atoppi]

Looking further into this, it seems that a publisher stream video profile (ps->h264_profile) is being saved only in case a videoroom profile has been configured (and also the configured profile matches the publisher's offer).
Since ps->h264_profile is the profile that will be offered to subscribers, this explains the missing fmtp.

[atoppi]

hi @sdroege , can you please test the attached patch on top of latest multistream ?

git apply fix_multistream_vprofile.txt

Then build and restart janus.
fix_multistream_vprofile.txt

[lminiero]

@sdroege did you have a chance to check if the patch Alessandro prepared fixes it for you too? It seems to work for us, so unless you tell us it still needs fixing we'll push it to the branch.

[sdroege]

I had no chance to test it yet, sorry. It looks like it should work to me, so feel free to push it. If something is still broken once I get to test it, I'll let you know!

[lminiero]

Ack, thanks! We'll push the fix and close the issue for the time being then 👍