Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multistream support via Unified Plan #1459

Closed
wants to merge 146 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
146 commits
Select commit Hold shift + click to select a range
571a0e1
Initial refactoring of internal handle/peerconnection core structures
lminiero Nov 23, 2018
d4ce7c4
Started making SDP code more flexible
lminiero Nov 26, 2018
774ad17
Added missing g_error_free calls where needed (see #1445)
lminiero Nov 27, 2018
363c3ce
Cleaner Admin API info
lminiero Nov 27, 2018
24f04b2
Streamlined a few outgoing SDP attributes
lminiero Nov 27, 2018
baf90d2
Differentiated data from handshake in packets/bytes stats
lminiero Nov 27, 2018
063e035
Modified media routing signatures to involve mindex (in plugins too)
lminiero Nov 27, 2018
5a0fd5b
Refactored the SDP utils to support multiple m-lines
lminiero Nov 28, 2018
7f41cd7
Fixed parsing of SDPs with multiple m-lines
lminiero Nov 28, 2018
e1de6c2
Generate more compact mids
lminiero Nov 29, 2018
18c5a81
Fixed broken data channels in Firefox after SDP refactoring
lminiero Nov 29, 2018
0daf2d5
Force unified-plan sdpSemantics in janus.js
lminiero Nov 29, 2018
05acdd8
Added check to see if browser supports Unified Plan
lminiero Nov 29, 2018
c92b0ca
Added option to specify mid for mline when generating offer
lminiero Nov 30, 2018
9ab9563
Streamlined RTP switching context structure
lminiero Nov 30, 2018
f2d6ae9
First take at refactoring of Streaming plugin (still WIP)
lminiero Dec 4, 2018
ba9895e
Don't filter out mid attribute when anonymizing SDP
lminiero Dec 4, 2018
7dba389
More consistent usage of JANUS_JSON_* properties
lminiero Dec 4, 2018
93a200e
Allow plugins to set mid in SDP offers
lminiero Dec 4, 2018
db455c2
Added sample gstreamer script to feed multistream mountpoint
lminiero Dec 4, 2018
05a7bfb
Fixed detection of new/active stream in mountpoints
lminiero Dec 4, 2018
8976f82
Moved onlocalstream/onremotestream to onlocaltrack/onremotetrack in j…
lminiero Dec 4, 2018
1e5ad5b
Added additional gstreamer script samples to Makefile
lminiero Dec 4, 2018
7f23ef1
Fixed file-based mountpoints not working anymore
lminiero Dec 4, 2018
4f123b7
Better UI management of audio-only remote streams
lminiero Dec 4, 2018
dba43d2
Added concept of streams (and related mountpoint streams mapping) to …
lminiero Dec 4, 2018
18d4c5f
Notify manually removed local tracks
lminiero Dec 5, 2018
85a9619
Updated some other demos to the new onlocaltrack/onremotetrack in jan…
lminiero Dec 5, 2018
7f338ef
Ignore RTCP if it contains no SSRC
lminiero Dec 6, 2018
14305bb
Fixed broken VideoRoom subscription (wrong RTP context)
lminiero Dec 6, 2018
12e4bbc
Updated VideoRoom-related demos to the new janus.js callbacks
lminiero Dec 6, 2018
30580af
Several small tweaks to demos after callbacks changes
lminiero Dec 6, 2018
bda1542
Small addition to SDP utils
lminiero Dec 10, 2018
7d2affb
First bulk of changes to VideoRoom plugin to switch to streams (still…
lminiero Dec 10, 2018
12c2c60
Fixed conflicts with master
lminiero Dec 11, 2018
97d73e5
Fixed many locks/references/typos in VideoRoom plugin
lminiero Dec 11, 2018
02bf2d3
First take at refactoring VideoRoom subscribers API for multistream
lminiero Dec 12, 2018
79a03e6
Added API to unsubscribe from streams in the VideoRoom
lminiero Dec 13, 2018
88b7174
Wrap onlocaltrack and onremotetrack calls in try/catch to ignore app …
lminiero Dec 17, 2018
477144d
Remove brackets from local track.id when using it to index HTML elements
lminiero Dec 17, 2018
743bd43
Removed commented debug code
lminiero Dec 17, 2018
94d14e9
Added API to subscribe to streams after setup in the VideoRoom
lminiero Dec 17, 2018
9f8c5f7
Ensure msid attributes in the SDP are unique
lminiero Dec 17, 2018
5282580
Fixed occasional deadlock when hanging up VideoRoom stream
lminiero Dec 18, 2018
6dfc954
Post-pone offer update if we're still waiting for an answer from a su…
lminiero Dec 18, 2018
227a3bf
Make sure we don't send media on a stream until it has been answered to
lminiero Dec 20, 2018
96745f8
Handle onmute event on remote tracks
lminiero Dec 20, 2018
046306c
Don't modify the send property when removing/replacing streams
lminiero Dec 20, 2018
2349602
Fixed segfault in VideoRoom when publisher leaves
lminiero Dec 20, 2018
516f67e
Send a new offer to all subscribers after a publisher leaves
lminiero Dec 20, 2018
67f5c19
Reset local SSRC after a medium is set to inactive
lminiero Dec 20, 2018
f111315
Added new demo page to test VideoRoom with multistream
lminiero Dec 20, 2018
c0af7ab
Aligned to recent changes in master
lminiero Dec 21, 2018
6e0ed7c
Fixed typo
lminiero Dec 21, 2018
8068462
Better management of simulcasting
lminiero Dec 21, 2018
3fa3af7
Fixed some warnings
lminiero Dec 21, 2018
ccb3c94
Fixed missing code for resuming media via configure
lminiero Dec 21, 2018
78a6049
Cleaned up code invoking PLI without helper
lminiero Dec 21, 2018
11dff11
Added mid to media (receiving audio/video) API events
lminiero Jan 7, 2019
a01e82e
Started to update docs: onlocaltrack/onremotetrack
lminiero Jan 7, 2019
0abf799
Added new API for VideoRoom RTP forwarders (old API still there but d…
lminiero Jan 7, 2019
7e72fa4
Aligned to latest master
lminiero Jan 8, 2019
e6954d9
Implemented partial subscriptions in Streaming plugin
lminiero Jan 8, 2019
8c972d1
Added VideoRoom publisher API to add descriptions to published streams
lminiero Jan 8, 2019
ee487d5
Added mid to media (receiving audio/video) ev.handlers events as well
lminiero Jan 8, 2019
733ba50
Added global map for VideoRoom RTP forwarders indexes
lminiero Jan 9, 2019
66692d1
Updated getVolume and related methods in janus.js
lminiero Jan 9, 2019
865b5db
Updated mute/unmute methods in janus.js to accept optional mid
lminiero Jan 9, 2019
af82edc
Updated getBitrate method in janus.js to accept optional mid
lminiero Jan 9, 2019
ed98527
Aligned to latest fixes in master
lminiero Jan 10, 2019
1530588
Fixed conflicts with master
lminiero Jan 22, 2019
fe4c5d0
Better management of Chrome <72 when renegotiating in janus.js
lminiero Jan 23, 2019
4350a5d
Started updating documentation for Streaming plugin
lminiero Jan 23, 2019
2ba7820
Merged libconfig fixes from master
lminiero Jan 23, 2019
35344d3
Added more updates to Streaming plugin documentation
lminiero Jan 23, 2019
dd0d67b
Initial update to the VideoRoom switch functionality (still WIP)
lminiero Feb 5, 2019
b839428
Aligned to latest changes in master
lminiero Feb 5, 2019
d3d24a9
Use booleans when serializing atomic flags to JSON in plugins
lminiero Feb 6, 2019
37a966a
Added onunmute handler for tracks in janus.js
lminiero Feb 6, 2019
8e2c0b7
Updated VideoRoom documentation
lminiero Feb 6, 2019
c8bb35a
More updates to Streaming plugin docs and demo
lminiero Feb 11, 2019
519f0a1
Prevent multiple watch requests because of already bound click function
lminiero Feb 11, 2019
34820aa
Fixed a couple of bugs in the VideoRoom demos
lminiero Feb 12, 2019
6ff2199
Fixed compilation issue when libcurl is not available
lminiero Feb 20, 2019
a7ff9db
Fixed other compilation issue when libcurl is not available
lminiero Feb 21, 2019
a5c3692
Fixed crash when hanging up and using datachannels
lminiero Feb 26, 2019
d71a298
Aligned to latest master and fixed related conflicts
lminiero Feb 26, 2019
37c143d
Made VideoCall plugin media routing multistream-aware
lminiero Feb 26, 2019
adfb1b1
Fixed mindex when relaying RTCP in EchoTest
lminiero Feb 26, 2019
2981a7c
Aligned to latest fixes in master
lminiero Mar 4, 2019
e938143
Fixed broken SDP negotiation of RTP extensions in VideoRoom
lminiero Mar 4, 2019
64640d8
Fixed some more RTP extensions negotiations in VideoRoom
lminiero Mar 4, 2019
c6c2f37
Use larger buffer when merging SDPs for WebRTC (fixes #1534)
lminiero Mar 6, 2019
70b4d3f
Use larger buffer by default for SDP (see #1534)
lminiero Mar 7, 2019
510805b
Aligned with master and fixed conflicts
lminiero Mar 7, 2019
96febf8
Send a PLI to the new publisher when switching
lminiero Mar 11, 2019
c88fa4a
Added way to specify substream/spatial/temporal layer target at subsc…
lminiero Mar 11, 2019
057bc4e
Restored (best-effort) legacy switch syntax in VideoRoom
lminiero Mar 11, 2019
78d936f
Merged support for mid RTP extension
lminiero Mar 12, 2019
2052c70
Added basic rid extension management
lminiero Mar 12, 2019
e53f64d
Fixed occasional crash when switching in VideoRoom (fixes #1541)
lminiero Mar 12, 2019
eee1782
Aligned to latest fixes in master, and fixed related conflicts
lminiero Mar 15, 2019
e61512b
Aligned with new rid-based simulcast changes in master (WIP)
lminiero Mar 18, 2019
32ae35c
Fixed conflicts with master and fixed a couple of typos
lminiero Mar 19, 2019
4bd190a
Imported fix on simulcast layer changes from master
lminiero Mar 19, 2019
7f66b8f
Imported new support for multiple streams in datachannels
lminiero Mar 20, 2019
4c9bbdd
Use labels to address data messages from publishers in VideoRoom subs…
lminiero Mar 20, 2019
6a54e28
Fixed typo
lminiero Mar 22, 2019
a0240a1
Fixed missing unlocks when trying to add a new datachannel to a subsc…
lminiero Apr 1, 2019
43aeb91
Removed extra (and unneeded) unlock that could cause segfaults
lminiero Apr 16, 2019
63182d0
Aligned to latest changes in master and fixed conflicts
lminiero Apr 16, 2019
da8c2ae
Fixed conflict with recent change in master
lminiero Apr 19, 2019
cf7e932
ice: check if rtcp_ctx is NULL before use (#1601)
tmatth May 1, 2019
fd7ca27
Merged some more fixes from master and fixed related conflicts
lminiero May 1, 2019
35e9178
Fixed conflict
lminiero May 2, 2019
24270a8
Merge branch 'master' into unified-plan
lminiero May 3, 2019
e36699b
Aligned to latest changes in master, and fixed related conflicts
lminiero May 15, 2019
7776508
Added error messages to some catch blocks in janus.js
lminiero May 15, 2019
0f4a319
Fixed typo
lminiero May 15, 2019
76a838f
Fixed exceptions in demos trying to query non-existing video element …
lminiero May 15, 2019
3539a9e
Fixed broken video in multistream demo after unmute/mute of remote track
lminiero May 15, 2019
45a88b0
Make sure bitrate timer is removed when unsubscribing from a feed
lminiero May 15, 2019
960a5c6
Fixed typo in VideoRoom plugin
lminiero May 15, 2019
e0728f8
Fixed typos in Streaming plugin (see #1629)
lminiero May 17, 2019
76098c9
Fixed broken management of datachannel subscriptions in VideoRoom
lminiero May 20, 2019
550135a
Fixed conflicts with master
lminiero May 20, 2019
ef1e3af
Fixed typo
lminiero May 20, 2019
c6b8b2a
Fixed wrong size of video elements in VideoRoom demos
lminiero May 20, 2019
9beba3c
Fixed reference count leak when using datachannels
lminiero May 22, 2019
b157a72
Fixed exception in janus.js when using datachannels
lminiero May 22, 2019
164c676
Fixed wrong size of video box in EchoTest demos
lminiero May 22, 2019
a1c5b10
Aligned to latest changes and fixes in master
lminiero Jun 7, 2019
1715d45
Added mid info to mediaState event in janus.js
lminiero Jun 10, 2019
75ed765
Aligned to changes in Streaming plugin, and fixed broken RTSP support
lminiero Jun 14, 2019
39a9960
Aligned to latest changes and fixes in master
lminiero Jun 24, 2019
2fbd6a1
Updated unified-plan branch to use the new slowLink approach
lminiero Jun 25, 2019
fd14e16
Fixed conflicts with master
lminiero Jun 25, 2019
a4ed127
Merged SSRC-related fixes and fixed conflicts
lminiero Jun 25, 2019
4b2caa7
Fixed conflict
lminiero Jun 27, 2019
a0300e4
Don't add ssrc-related lines in the SDP if SSRC is 0
lminiero Jul 11, 2019
66bff68
Fixed segfault when usrsctp is not available (see #1700)
lminiero Jul 17, 2019
ad021ad
Fixed conflicts with master afte recent fixes
lminiero Jul 17, 2019
fb4e6f4
plugins: drop dead code related to rtp_header_update (#1715)
tmatth Jul 26, 2019
0945b1f
Aligned to version 0.7.5
lminiero Sep 6, 2019
b5f0b6d
Fixed simulcast-related typos
lminiero Oct 1, 2019
a8a1689
Aligned to new version 0.7.6 and fixed conflicts
lminiero Oct 29, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -446,7 +446,9 @@ stream_DATA += \
plugins/streams/music.mulaw \
plugins/streams/radio.alaw \
plugins/streams/test_gstreamer.sh \
plugins/streams/test_gstreamer_1.sh
plugins/streams/test_gstreamer1.sh \
plugins/streams/test_gstreamer_multistream.sh \
plugins/streams/test_gstreamer1_multistream.sh
EXTRA_DIST += \
conf/janus.plugin.streaming.jcfg.sample.in \
$(stream_DATA)
Expand Down
53 changes: 53 additions & 0 deletions conf/janus.plugin.streaming.jcfg.sample.in
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,7 @@ file-ondemand-sample: {
#videopt = 126
#videortpmap = "H264/90000"
#videofmtp = "profile-level-id=42e01f;packetization-mode=1"
#secret = "adminpwd"
#}

#
Expand All @@ -162,6 +163,7 @@ file-ondemand-sample: {
#video = false
#data = true
#dataport = 5008
#secret = "adminpwd"
#}

#
Expand All @@ -182,6 +184,7 @@ file-ondemand-sample: {
#videomcast = "232.3.4.5"
#videopt = 100
#videortpmap = "VP8/90000"
#secret = "adminpwd"
#}

#
Expand All @@ -202,4 +205,54 @@ file-ondemand-sample: {
#url = "rtsp://127.0.0.1:8554/unicast"
#rtsp_user = "username"
#rtsp_pwd = "password"
#secret = "adminpwd"
#}

#
# Finally, this is an example of how you can configure a live Streaming
# mountpoint to send multiple streams of the same type at the same time:
# that is, not simulcasting, but different streams (e.g., two audio
# streams and two video streams). To do so, you don't set the audio,
# video and data properties inline, but use an array of properties instead,
# each identifying a single stream to add, that will then translate to
# a dedicated m-line in the SDP. For each stream, you specify the type,
# a unique ID (mid), and can provide a short description (label) so that
# the client side can know what's what when rendering the streams. Notice
# how the port/pt/rtpmap/fmtp/etc. stuff is called just like that, without
# any audio/video/data prefix: in fact, each media stream can be configured
# the same way, and it's the type that allows us to differentiate them.
# As such, you can use the same approach for creating regular mountpoints
# as well (e.g., 1 audio and 1 video) in a much clearer, and cleaner, way.
#
multistream-test: {
type = "rtp"
id = 123
description = "Multistream test (1 audio, 2 video)"
media = (
{
type = "audio"
mid = "a"
label = "Audio stream"
port = 5102
pt = 111
rtpmap = "opus/48000/2"
},
{
type = "video"
mid = "v1"
label = "Video stream #1"
port = 5104
pt = 100
rtpmap = "VP8/90000"
},
{
type = "video"
mid = "v2"
label = "Video stream #2"
port = 5106
pt = 100
rtpmap = "VP8/90000"
}
)
secret = "adminpwd"
}
25 changes: 10 additions & 15 deletions dtls-bio.c
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
* \copyright GNU General Public License v3
* \brief OpenSSL BIO agent writer
* \details OpenSSL BIO that writes packets to a libnice agent.
*
*
* \ingroup protocols
* \ref protocols
*/
Expand Down Expand Up @@ -110,7 +110,7 @@ static int janus_dtls_bio_agent_free(BIO *bio) {
#endif
return 1;
}

static int janus_dtls_bio_agent_write(BIO *bio, const char *in, int inl) {
JANUS_LOG(LOG_HUGE, "janus_dtls_bio_agent_write: %p, %d\n", in, inl);
/* Forward data to the write BIO */
Expand All @@ -129,17 +129,12 @@ static int janus_dtls_bio_agent_write(BIO *bio, const char *in, int inl) {
JANUS_LOG(LOG_ERR, "No DTLS-SRTP stack, no DTLS bridge...\n");
return -1;
}
janus_ice_component *component = (janus_ice_component *)dtls->component;
if(component == NULL) {
JANUS_LOG(LOG_ERR, "No component, no DTLS bridge...\n");
return -1;
}
janus_ice_stream *stream = component->stream;
if(!stream) {
JANUS_LOG(LOG_ERR, "No stream, no DTLS bridge...\n");
janus_handle_webrtc *pc = (janus_handle_webrtc *)dtls->pc;
if(pc == NULL) {
JANUS_LOG(LOG_ERR, "No WebRTC PeerConnection, no DTLS bridge...\n");
return -1;
}
janus_ice_handle *handle = stream->handle;
janus_handle *handle = pc->handle;
if(!handle || !handle->agent || !dtls->write_bio) {
JANUS_LOG(LOG_ERR, "No handle/agent/bio, no DTLS bridge...\n");
return -1;
Expand All @@ -149,17 +144,17 @@ static int janus_dtls_bio_agent_write(BIO *bio, const char *in, int inl) {
/* FIXME Just a warning for now, this will need to be solved with proper fragmentation */
JANUS_LOG(LOG_WARN, "[%"SCNu64"] The DTLS stack is trying to send a packet of %d bytes, this may be larger than the MTU and get dropped!\n", handle->handle_id, inl);
}
int bytes = nice_agent_send(handle->agent, component->stream_id, component->component_id, inl, in);
int bytes = nice_agent_send(handle->agent, pc->stream_id, pc->component_id, inl, in);
if(bytes < inl) {
JANUS_LOG(LOG_ERR, "[%"SCNu64"] Error sending DTLS message on component %d of stream %d (%d)\n", handle->handle_id, component->component_id, stream->stream_id, bytes);
JANUS_LOG(LOG_ERR, "[%"SCNu64"] Error sending DTLS message on component %d of stream %d (%d)\n", handle->handle_id, pc->component_id, pc->stream_id, bytes);
} else {
JANUS_LOG(LOG_HUGE, "[%"SCNu64"] >> >> ... and sent %d of those bytes on the socket\n", handle->handle_id, bytes);
}
/* Update stats (TODO Do the same for the last second window as well)
* FIXME: the Data stats includes the bytes used for the handshake */
if(bytes > 0) {
component->out_stats.data.packets++;
component->out_stats.data.bytes += bytes;
pc->dtls_out_stats.info[0].packets++;
pc->dtls_out_stats.info[0].bytes += bytes;
}
return bytes;
}
Expand Down
Loading