-
Notifications
You must be signed in to change notification settings - Fork 2.5k
/
janus_videoroom.c
13822 lines (13656 loc) · 614 KB
/
janus_videoroom.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
/*! \file janus_videoroom.c
* \author Lorenzo Miniero <[email protected]>
* \copyright GNU General Public License v3
* \brief Janus VideoRoom plugin
* \details Check the \ref videoroom for more details.
*
* \ingroup plugins
* \ref plugins
*
* \page videoroom VideoRoom plugin documentation
* This is a plugin implementing a videoconferencing SFU
* (Selective Forwarding Unit) for Janus, that is an audio/video router.
* This means that the plugin implements a virtual conferencing room peers
* can join and leave at any time. This room is based on a Publish/Subscribe
* pattern. Each peer can publish his/her own live audio/video feeds: this
* feed becomes an available stream in the room the other participants can
* subscribe to. This means that this plugin allows the realization of several
* different scenarios, ranging from a simple webinar (one speaker, several
* watchers) to a fully meshed video conference (each peer sending and
* receiving to and from all the others).
*
* Notice that, since Janus now supports multistream PeerConnections,
* subscriptions can be done either in "bulks" (you use a single PeerConnection
* to subscribe to multiple streams from one or more publishers) or
* separately (each PeerConnections represents a subscription to a single
* publisher). Same thing for publishers: you may choose to publish, e.g.,
* audio and video on one PeerConnection, and share your screen on another,
* or publish everything on the same PeerConnection instead. While
* functionally both approaches (multistream vs. legacy mode) are the same
* (the same media flows in both cases), the differences are in how
* resources are used, and in how the client has to handle incoming and
* outgoing connections. Besides, one approach might make more sense in
* some scenarios, and the other make more sense in different use cases.
* As such, the approach to follow is left to the developer and the application.
*
* What is important to point out, though, is that publishers and subscribers
* will in all cases require different PeerConnections. This means that,
* even with multistream, you won't be able to use a single PeerConnection
* to send your contributions and receive those from everyone else. This
* is a choice done by design, to avoid the issues that would inevitably
* arise when doing, for instance, renegotiations to update the streams.
*
* On a more general note and to give some more context with respect to the
* core functionality in Janus, notice that, considering this plugin allows
* for several different WebRTC PeerConnections to be on at the same time
* for the same peer (different publishers and subscribers for sure, and
* potentially more than one of each if multistream is not in use), each
* peer will often need to attach several times to the same plugin for each
* stream: this means that each peer needs to have at least one handle active
* for managing its relation with the plugin (joining a room,
* leaving a room, muting/unmuting, publishing, receiving events), and needs
* to open others when they want to subscribe to a feed from other participants
* (the number depends on the subscription approach of choice). Handles
* used for subscriptions, though, would be logically "subjects" to the
* master one used for managing the room: this means that they cannot be
* used, for instance, to unmute in the room, as their only purpose would
* be to provide a context in which creating the recvonly PeerConnections
* for the subscription(s).
*
* Rooms to make available are listed in the plugin configuration file.
* A pre-filled configuration file is provided in \c conf/janus.plugin.videoroom.jcfg
* and includes a demo room for testing. The same plugin is also used
* dynamically (that is, with rooms created on the fly via API) in the
* Screen Sharing demo as well.
*
* To add more rooms or modify the existing one, you can use the following
* syntax:
*
* \verbatim
room-<unique room ID>: {
description = This is my awesome room
is_private = true|false (private rooms don't appear when you do a 'list' request, default=false)
secret = <optional password needed for manipulating (e.g. destroying) the room>
pin = <optional password needed for joining the room>
require_pvtid = true|false (whether subscriptions are required to provide a valid private_id
to associate with a publisher, default=false)
signed_tokens = true|false (whether access to the room requires signed tokens; default=false,
only works if signed tokens are used in the core as well)
publishers = <max number of concurrent senders> (e.g., 6 for a video
conference or 1 for a webinar, default=3)
bitrate = <max video bitrate for senders> (e.g., 128000)
bitrate_cap = <true|false, whether the above cap should act as a limit to dynamic bitrate changes by publishers, default=false>,
fir_freq = <send a FIR to publishers every fir_freq seconds> (0=disable)
audiocodec = opus|g722|pcmu|pcma|isac32|isac16 (audio codec to force on publishers, default=opus
can be a comma separated list in order of preference, e.g., opus,pcmu)
videocodec = vp8|vp9|h264|av1|h265 (video codec to force on publishers, default=vp8
can be a comma separated list in order of preference, e.g., vp9,vp8,h264)
vp9_profile = VP9-specific profile to prefer (e.g., "2" for "profile-id=2")
h264_profile = H.264-specific profile to prefer (e.g., "42e01f" for "profile-level-id=42e01f")
opus_fec = true|false (whether inband FEC must be negotiated; only works for Opus, default=true)
opus_dtx = true|false (whether DTX must be negotiated; only works for Opus, default=false)
audiolevel_ext = true|false (whether the ssrc-audio-level RTP extension must be
negotiated/used or not for new publishers, default=true)
audiolevel_event = true|false (whether to emit event to other users or not, default=false)
audio_active_packets = 100 (number of packets with audio level, default=100, 2 seconds)
audio_level_average = 25 (average value of audio level, 127=muted, 0='too loud', default=25)
videoorient_ext = true|false (whether the video-orientation RTP extension must be
negotiated/used or not for new publishers, default=true)
playoutdelay_ext = true|false (whether the playout-delay RTP extension must be
negotiated/used or not for new publishers, default=true)
transport_wide_cc_ext = true|false (whether the transport wide CC RTP extension must be
negotiated/used or not for new publishers, default=true)
record = true|false (whether this room should be recorded, default=false)
rec_dir = <folder where recordings should be stored, when enabled>
lock_record = true|false (whether recording can only be started/stopped if the secret
is provided, or using the global enable_recording request, default=false)
notify_joining = true|false (optional, whether to notify all participants when a new
participant joins the room. The Videoroom plugin by design only notifies
new feeds (publishers), and enabling this may result extra notification
traffic. This flag is particularly useful when enabled with require_pvtid
for admin to manage listening only participants. default=false)
require_e2ee = true|false (whether all participants are required to publish and subscribe
using end-to-end media encryption, e.g., via Insertable Streams; default=false)
dummy_publisher = true|false (whether a dummy publisher should be created in this room,
with one separate m-line for each codec supported in the room; this is
useful when there's a need to create subscriptions with placeholders
for some or all m-lines, even when they aren't used yet; default=false)
dummy_streams = in case dummy_publisher is set to true, array of codecs to offer,
optionally with a fmtp attribute to match (codec/fmtp properties).
If not provided, all codecs enabled in the room are offered, with no fmtp.
Notice that the fmtp is parsed, and only a few codecs are supported.
threads = number of threads to assist with the relaying of publishers in the room; as
in the Streaming plugin, this setting can help if you expect a lot of subscribers
that may cause the plugin to slow down and fail to catch up (default=0)
}
\endverbatim
*
* Note that recording will work with all codecs except iSAC.
*
* \section sfuapi Video Room API
*
* The Video Room API supports several requests, some of which are
* synchronous and some asynchronous. There are some situations, though,
* (invalid JSON, invalid request) which will always result in a
* synchronous error response even for asynchronous requests.
*
* \c create , \c destroy , \c edit , \c exists, \c list, \c allowed,
* \c kick , \c moderate , \c enable_recording , \c listparticipants
* and \c listforwarders are synchronous requests, which means you'll
* get a response directly within the context of the transaction.
* \c create allows you to create a new video room dynamically, as an
* alternative to using the configuration file; \c edit allows you to
* dynamically edit some room properties (e.g., the PIN); \c destroy removes a
* video room and destroys it, kicking all the users out as part of the
* process; \c exists allows you to check whether a specific video room
* exists; finally, \c list lists all the available rooms, while \c
* listparticipants lists all the active (as in currently publishing
* something) participants of a specific room and their details.
*
* The \c join , \c joinandconfigure , \c configure , \c publish ,
* \c unpublish , \c start , \c pause , \c switch and \c leave
* requests instead are all asynchronous, which
* means you'll get a notification about their success or failure in
* an event. \c join allows you to join a specific video room, specifying
* whether that specific PeerConnection will be used for publishing or
* watching; \c configure can be used to modify some of the participation
* settings (e.g., bitrate cap); \c joinandconfigure combines the previous
* two requests in a single one (just for publishers); \c publish can be
* used to start sending media to broadcast to the other participants,
* while \c unpublish does the opposite; \c start allows you to start
* receiving media from a publisher you've subscribed to previously by
* means of a \c join , while \c pause pauses the delivery of the media;
* the \c switch request can be used to change the source of the media
* flowing over a specific PeerConnection (e.g., I was watching Alice,
* I want to watch Bob now) without having to create a new handle for
* that; finally, \c leave allows you to leave a video room for good
* (or, in the case of viewers, definitely closes a subscription).
*
* \c create can be used to create a new video room, and has to be
* formatted as follows:
*
\verbatim
{
"request" : "create",
"room" : <unique numeric ID, optional, chosen by plugin if missing>,
"permanent" : <true|false, whether the room should be saved in the config file, default=false>,
"description" : "<pretty name of the room, optional>",
"secret" : "<password required to edit/destroy the room, optional>",
"pin" : "<password required to join the room, optional>",
"is_private" : <true|false, whether the room should appear in a list request>,
"allowed" : [ array of string tokens users can use to join this room, optional],
...
}
\endverbatim
*
* For the sake of brevity, not all of the available settings are listed
* here. You can refer to the name of the properties in the configuration
* file as a reference, as the ones used to programmatically create a new
* room are exactly the same.
*
* A successful creation procedure will result in a \c created response:
*
\verbatim
{
"videoroom" : "created",
"room" : <unique numeric ID>,
"permanent" : <true if saved to config file, false if not>
}
\endverbatim
*
* If you requested a permanent room but a \c false value is returned
* instead, good chances are that there are permission problems.
*
* An error instead (and the same applies to all other requests, so this
* won't be repeated) would provide both an error code and a more verbose
* description of the cause of the issue:
*
\verbatim
{
"videoroom" : "event",
"error_code" : <numeric ID, check Macros below>,
"error" : "<error description as a string>"
}
\endverbatim
*
* Notice that, in general, all users can create rooms. If you want to
* limit this functionality, you can configure an admin \c admin_key in
* the plugin settings. When configured, only "create" requests that
* include the correct \c admin_key value in an "admin_key" property
* will succeed, and will be rejected otherwise. Notice that you can
* optionally extend this functionality to RTP forwarding as well, in
* order to only allow trusted clients to use that feature.
*
* Once a room has been created, you can still edit some (but not all)
* of its properties using the \c edit request. This allows you to modify
* the room description, secret, pin and whether it's private or not: you
* won't be able to modify other more static properties, like the room ID,
* the sampling rate, the extensions-related stuff and so on. If you're
* interested in changing the ACL, instead, check the \c allowed message.
* An \c edit request has to be formatted as follows:
*
\verbatim
{
"request" : "edit",
"room" : <unique numeric ID of the room to edit>,
"secret" : "<room secret, mandatory if configured>",
"new_description" : "<new pretty name of the room, optional>",
"new_secret" : "<new password required to edit/destroy the room, optional>",
"new_pin" : "<new password required to join the room, optional>",
"new_is_private" : <true|false, whether the room should appear in a list request>,
"new_require_pvtid" : <true|false, whether the room should require private_id from subscribers>,
"new_bitrate" : <new bitrate cap to force on all publishers (except those with custom overrides)>,
"new_fir_freq" : <new period for regular PLI keyframe requests to publishers>,
"new_publishers" : <new cap on the number of concurrent active WebRTC publishers>,
"new_lock_record" : <true|false, whether recording state can only be changed when providing the room secret>,
"new_rec_dir" : "<the new path where the next .mjr files should being saved>",
"permanent" : <true|false, whether the room should be also removed from the config file, default=false>
}
\endverbatim
*
* A successful edit procedure will result in an \c edited response:
*
\verbatim
{
"videoroom" : "edited",
"room" : <unique numeric ID>
}
\endverbatim
*
* On the other hand, \c destroy can be used to destroy an existing video
* room, whether created dynamically or statically, and has to be
* formatted as follows:
*
\verbatim
{
"request" : "destroy",
"room" : <unique numeric ID of the room to destroy>,
"secret" : "<room secret, mandatory if configured>",
"permanent" : <true|false, whether the room should be also removed from the config file, default=false>
}
\endverbatim
*
* A successful destruction procedure will result in a \c destroyed response:
*
\verbatim
{
"videoroom" : "destroyed",
"room" : <unique numeric ID>
}
\endverbatim
*
* This will also result in a \c destroyed event being sent to all the
* participants in the video room, which will look like this:
*
\verbatim
{
"videoroom" : "destroyed",
"room" : <unique numeric ID of the destroyed room>
}
\endverbatim
*
* You can check whether a room exists using the \c exists request,
* which has to be formatted as follows:
*
\verbatim
{
"request" : "exists",
"room" : <unique numeric ID of the room to check>
}
\endverbatim
*
* A successful request will result in a \c success response:
*
\verbatim
{
"videoroom" : "success",
"room" : <unique numeric ID>,
"exists" : <true|false>
}
\endverbatim
*
* You can configure whether to check tokens or add/remove people who can join
* a room using the \c allowed request, which has to be formatted as follows:
*
\verbatim
{
"request" : "allowed",
"secret" : "<room secret, mandatory if configured>",
"action" : "enable|disable|add|remove",
"room" : <unique numeric ID of the room to update>,
"allowed" : [
// Array of strings (tokens users might pass in "join", only for add|remove)
]
}
\endverbatim
*
* A successful request will result in a \c success response:
*
\verbatim
{
"videoroom" : "success",
"room" : <unique numeric ID>,
"allowed" : [
// Updated, complete, list of allowed tokens (only for enable|add|remove)
]
}
\endverbatim
*
* If you're the administrator of a room (that is, you created it and have access
* to the secret) you can kick participants using the \c kick request. Notice
* that this only kicks the user out of the room, but does not prevent them from
* re-joining: to ban them, you need to first remove them from the list of
* authorized users (see \c allowed request) and then \c kick them. The \c kick
* request has to be formatted as follows:
*
\verbatim
{
"request" : "kick",
"secret" : "<room secret, mandatory if configured>",
"room" : <unique numeric ID of the room>,
"id" : <unique numeric ID of the participant to kick>
}
\endverbatim
*
* A successful request will result in a \c success response:
*
\verbatim
{
"videoroom" : "success",
}
\endverbatim
*
* As an administrator, you can also forcibly mute/unmute any of the media
* streams sent by participants (i.e., audio, video and data streams),
* using the \c moderate requests. Notice that if the participant is self
* muted on a stream, and you unmute that stream with \c moderate, they
* will NOT be unmuted: you'll simply remove any moderation block
* that may have been enforced on the participant for that medium
* themselves. The \c moderate request has to be formatted as follows:
*
\verbatim
{
"request" : "moderate",
"secret" : "<room secret, mandatory if configured>",
"room" : <unique numeric ID of the room>,
"id" : <unique numeric ID of the participant to moderate>,
"mid" : <mid of the m-line to refer to for this moderate request>,
"mute" : <true|false, depending on whether the media addressed by the above mid should be muted by the moderator>
}
\endverbatim
*
* A successful request will result in a \c success response:
*
\verbatim
{
"videoroom" : "success",
}
\endverbatim
*
* To get a list of the available rooms you can make use of the \c list request.
* \c admin_key is optional. If included and correct, rooms configured/created
* as private will be included in the list as well.
*
\verbatim
{
"request" : "list"
}
\endverbatim
*
* A successful request will produce a list of rooms in a \c success response:
*
\verbatim
{
"videoroom" : "success",
"list" : [ // Array of room objects
{ // Room #1
"room" : <unique numeric ID>,
"description" : "<Name of the room>",
"pin_required" : <true|false, whether a PIN is required to join this room>,
"is_private" : <true|false, whether this room is 'private' (as in hidden) or not>,
"max_publishers" : <how many publishers can actually publish via WebRTC at the same time>,
"bitrate" : <bitrate cap that should be forced (via REMB) on all publishers by default>,
"bitrate_cap" : <true|false, whether the above cap should act as a limit to dynamic bitrate changes by publishers (optional)>,
"fir_freq" : <how often a keyframe request is sent via PLI/FIR to active publishers>,
"require_pvtid": <true|false, whether subscriptions in this room require a private_id>,
"require_e2ee": <true|false, whether end-to-end encrypted publishers are required>,
"dummy_publisher": <true|false, whether a dummy publisher exists for placeholder subscriptions>,
"notify_joining": <true|false, whether an event is sent to notify all participants if a new participant joins the room>,
"audiocodec" : "<comma separated list of allowed audio codecs>",
"videocodec" : "<comma separated list of allowed video codecs>",
"opus_fec": <true|false, whether inband FEC must be negotiated (note: only available for Opus) (optional)>,
"opus_dtx": <true|false, whether DTX must be negotiated (note: only available for Opus) (optional)>,
"record" : <true|false, whether the room is being recorded>,
"rec_dir" : "<if recording, the path where the .mjr files are being saved>",
"lock_record" : <true|false, whether the room recording state can only be changed providing the secret>,
"num_participants" : <count of the participants (publishers, active or not; not subscribers)>
"audiolevel_ext": <true|false, whether the ssrc-audio-level extension must be negotiated or not for new publishers>,
"audiolevel_event": <true|false, whether to emit event to other users about audiolevel>,
"audio_active_packets": <amount of packets with audio level for checkup (optional, only if audiolevel_event is true)>,
"audio_level_average": <average audio level (optional, only if audiolevel_event is true)>,
"videoorient_ext": <true|false, whether the video-orientation extension must be negotiated or not for new publishers>,
"playoutdelay_ext": <true|false, whether the playout-delay extension must be negotiated or not for new publishers>,
"transport_wide_cc_ext": <true|false, whether the transport wide cc extension must be negotiated or not for new publishers>
},
// Other rooms
]
}
\endverbatim
*
* To get a list of the participants in a specific room, instead, you
* can make use of the \c listparticipants request, which has to be
* formatted as follows:
*
\verbatim
{
"request" : "listparticipants",
"room" : <unique numeric ID of the room>
}
\endverbatim
*
* A successful request will produce a list of participants in a
* \c participants response:
*
\verbatim
{
"videoroom" : "participants",
"room" : <unique numeric ID of the room>,
"participants" : [ // Array of participant objects
{ // Participant #1
"id" : <unique numeric ID of the participant>,
"display" : "<display name of the participant, if any; optional>",
"publisher" : "<true|false, whether user is an active publisher in the room>",
"talking" : <true|false, whether user is talking or not (only if audio levels are used)>
},
// Other participants
]
}
\endverbatim
*
* This covers almost all the synchronous requests. All the asynchronous requests,
* plus a couple of additional synchronous requests we'll cover later, refer
* to participants instead, namely on how they can publish, subscribe, or
* more in general manage the media streams they may be sending or receiving.
*
* Considering the different nature of publishers and subscribers in the room,
* and more importantly how you establish PeerConnections in the respective
* cases, their API requests are addressed in separate subsections.
*
* \subsection vroompub VideoRoom Publishers
*
* In a VideoRoom, publishers are those participant handles that are able
* (although may choose not to, more on this later) publish media in the
* room, and as such become feeds that you can subscribe to.
*
* To specify that a handle will be associated with a publisher, you must use
* the \c join request with \c ptype set to \c publisher (note that, as it
* will be explained later, you can also use \c joinandconfigure for the
* purpose). The exact syntax of the request is the following:
*
\verbatim
{
"request" : "join",
"ptype" : "publisher",
"room" : <unique ID of the room to join>,
"id" : <unique ID to register for the publisher; optional, will be chosen by the plugin if missing>,
"display" : "<display name for the publisher; optional>",
"token" : "<invitation token, in case the room has an ACL; optional>"
}
\endverbatim
*
* This will add the user to the list of participants in the room, although
* in a non-active role for the time being. Anyway, this participation
* allows the user to receive notifications about several aspects of the
* room on the related handle (including streams as they become available
* and go away). As such, it can be used even just as a way to get
* notifications in a room, without the need of ever actually publishing
* any stream at all (which explains why the "publisher" role may actually
* be a bit confusing in this context).
*
* A successful \c join will result in a \c joined event, which will contain
* a list of the currently active (as in publishing via WebRTC) publishers,
* and optionally a list of passive attendees (but only if the room was
* configured with \c notify_joining set to \c TRUE ):
*
\verbatim
{
"videoroom" : "joined",
"room" : <room ID>,
"description" : <description of the room, if available>,
"id" : <unique ID of the participant>,
"private_id" : <a different unique ID associated to the participant; meant to be private>,
"publishers" : [
{
"id" : <unique ID of active publisher #1>,
"display" : "<display name of active publisher #1, if any>",
"dummy" : <true if this participant is a dummy publisher>,
"streams" : [
{
"type" : "<type of published stream #1 (audio|video|data)">,
"mindex" : "<unique mindex of published stream #1>",
"mid" : "<unique mid of of published stream #1>",
"disabled" : <if true, it means this stream is currently inactive/disabled (and so codec, description, etc. will be missing)>,
"codec" : "<codec used for published stream #1>",
"description" : "<text description of published stream #1, if any>",
"moderated" : <true if this stream audio has been moderated for this participant>,
"simulcast" : "<true if published stream #1 uses simulcast>",
"svc" : "<true if published stream #1 uses SVC (VP9 and AV1 only)>",
"talking" : <true|false, whether the publisher stream has audio activity or not (only if audio levels are used)>,
},
// Other streams, if any
],
"talking" : <true|false, whether the publisher is talking or not (only if audio levels are used); deprecated, use the stream specific ones>,
},
// Other active publishers
],
"attendees" : [ // Only present when notify_joining is set to TRUE for rooms
{
"id" : <unique ID of attendee #1>,
"display" : "<display name of attendee #1, if any>"
},
// Other attendees
]
}
\endverbatim
*
* Notice that the publishers list will of course be empty if no one is
* currently active in the room. For what concerns the \c private_id
* property, it is meant to be used by the user when they create subscriptions,
* so that the plugin can associate subscriber handles (which are typically
* anonymous) to a specific participant; they're usually optional, unless
* required by the room configuration.
*
* As explained, with a simple \c join you're not an active publisher (there
* is no WebRTC PeerConnection yet), which means that by default your presence
* is not notified to other participants. In fact, the publish/subscribe nature
* of the plugin implies that by default only active publishers are notified,
* to allow participants to subscribe to existing feeds: notifying all joins/leaves,
* even those related to who will just lurk, may be overly verbose and chatty,
* especially in large rooms. Anyway, rooms can be configured to notify those
* as well, if the \c notify_joining property is set to true: in that case,
* regular joins will be notified too, in an event formatted like this:
*
\verbatim
{
"videoroom" : "event",
"room" : <room ID>,
"joining" : {
"id" : <unique ID of the new participant>,
"display" : "<display name of the new participant, if any>"
}
}
\endverbatim
*
* If you're interested in publishing media within a room, you can do that
* with a \c publish request. This request MUST be accompanied by a JSEP
* SDP offer to negotiate a new PeerConnection. The plugin will match it
* to the room configuration (e.g., to make sure the codecs you negotiated
* are allowed in the room), and will reply with a JSEP SDP answer to
* close the circle and complete the setup of the PeerConnection. As soon
* as the PeerConnection has been established, the publisher will become
* active, and a new active feed other participants can subscribe to.
*
* The syntax of a \c publish request is the following:
*
\verbatim
{
"request" : "publish",
"audiocodec" : "<audio codec to prefer among the negotiated ones; optional>",
"videocodec" : "<video codec to prefer among the negotiated ones; optional>",
"bitrate" : <bitrate cap to return via REMB; optional, overrides the global room value if present>,
"record" : <true|false, whether this publisher should be recorded or not; optional>,
"filename" : "<if recording, the base path/file to use for the recording files; optional>",
"display" : "<display name to use in the room; optional>",
"audio_level_average" : "<if provided, overrides the room audio_level_average for this user; optional>",
"audio_active_packets" : "<if provided, overrides the room audio_active_packets for this user; optional>",
"descriptions" : [ // Optional
{
"mid" : "<unique mid of a stream being published>",
"description" : "<text description of the stream (e.g., My front webcam)>"
},
// Other descriptions, if any
]}
\endverbatim
*
* As anticipated, since this is supposed to be accompanied by a JSEP SDP
* offer describing the publisher's media streams, the plugin will negotiate
* and prepare a matching JSEP SDP answer. Notice that, in principle, all
* published streams will be only identified by their unique \c mid and
* by their type (e.g., audio or video). In case you want to provide more
* information about the streams being published (e.g., to let other
* participants know that the first video is a camera, while the second
* video is a screen share), you can use the \c descriptions array for
* the purpose: each object in the array can be used to add a text description
* to associate to a specific mid, in order to help with the UI rendering.
* The \c descriptions property is optional, so no text will be provided
* by default: notice these descriptions can be updated dynamically via
* \c configure requests.
*
* If successful, a \c configured event will be sent back, formatted like this:
*
\verbatim
{
"videoroom" : "event",
"configured" : "ok"
}
\endverbatim
*
* This event will be accompanied by the prepared JSEP SDP answer.
*
* Notice that you can also use \c configure as a request instead of
* \c publish to start publishing. The two are functionally equivalent
* for publishing, but from a semantic perspective \c publish is the
* right message to send when publishing. The \c configure request, as
* it will be clearer later, can also be used to update some properties
* of the publisher session: in this case the \c publish request can NOT
* be used, as it can only be invoked to publish, and will fail if you're
* already publishing something.
*
* As an additional note, notice that you can also join and publish in
* a single request, which is useful in case you're not interested in
* first join as a passive attendee and only later publish something,
* but want to publish something right away. In this case you can use
* the \c joinandconfigure request, which as you can imagine combines
* the properties of both \c join and \c publish in a single request:
* the response to a \c joinandconfigure will be a \c joined event, and
* will again be accompanied by a JSEP SDP answer as usual.
*
* However you decided to publish something, as soon as the PeerConnection
* setup succeeds and the publisher becomes active, an event is sent to
* all the participants in the room with information on the new feed.
* The event must contain an array with a single element, and be formatted like this:
*
\verbatim
{
"videoroom" : "event",
"room" : <room ID>,
"publishers" : [
{
"id" : <unique ID of the new publisher>,
"display" : "<display name of the new publisher, if any>",
"dummy" : <true if this participant is a dummy publisher>,
"streams" : [
{
"type" : "<type of published stream #1 (audio|video|data)">,
"mindex" : "<unique mindex of published stream #1>",
"mid" : "<unique mid of of published stream #1>",
"disabled" : <if true, it means this stream is currently inactive/disabled (and so codec, description, etc. will be missing)>,
"codec" : "<codec used for published stream #1>",
"description" : "<text description of published stream #1, if any>",
"moderated" : <true if this stream audio has been moderated for this participant>,
"simulcast" : "<true if published stream #1 uses simulcast>",
"svc" : "<true if published stream #1 uses SVC (VP9 and AV1 only)>",
"talking" : <true|false, whether the publisher stream has audio activity or not (only if audio levels are used)>,
},
// Other streams, if any
],
"talking" : <true|false, whether the publisher is talking or not (only if audio levels are used); deprecated, use the stream specific ones>,
}
]
}
\endverbatim
*
* To stop publishing and tear down the related PeerConnection, you can
* use the \c unpublish request, which requires no arguments as the context
* is implicit:
*
\verbatim
{
"request" : "unpublish"
}
\endverbatim
*
* This will have the plugin tear down the PeerConnection, and remove the
* publisher from the list of active streams. If successful, the response
* will look like this:
*
\verbatim
{
"videoroom" : "event",
"unpublished" : "ok"
}
\endverbatim
*
* As soon as the PeerConnection is gone, all the other participants will
* also be notified about the fact that the stream is no longer available:
*
\verbatim
{
"videoroom" : "event",
"room" : <room ID>,
"unpublished" : <unique ID of the publisher who unpublished>
}
\endverbatim
*
* Notice that the same event will also be sent whenever the publisher
* feed disappears for reasons other than an explicit \c unpublish , e.g.,
* because the handle was closed or the user lost their connection.
* Besides, notice that you can publish and unpublish multiple times
* within the context of the same publisher handle.
*
* As anticipated above, you can use a request called \c configure to
* tweak some of the properties of an active publisher session. This
* request must be formatted as follows:
*
\verbatim
{
"request" : "configure",
"bitrate" : <bitrate cap to return via REMB; optional, overrides the global room value if present (unless bitrate_cap is set)>,
"keyframe" : <true|false, whether we should send this publisher a keyframe request>,
"record" : <true|false, whether this publisher should be recorded or not; optional>,
"filename" : "<if recording, the base path/file to use for the recording files; optional>",
"display" : "<new display name to use in the room; optional>",
"audio_active_packets" : "<new audio_active_packets to overwrite in the room one; optional>",
"audio_level_average" : "<new audio_level_average to overwrite the room one; optional>",
"streams" : [
{
"mid" : <mid of the m-line to tweak>,
"keyframe" : <true|false, whether we should send this stream a keyframe request; optional>,
"send" : <true|false, depending on whether the media addressed by the above mid should be relayed or not; optional>,
"min_delay" : <minimum delay to enforce via the playout-delay RTP extension, in blocks of 10ms; optional>,
"max_delay" : <maximum delay to enforce via the playout-delay RTP extension, in blocks of 10ms; optional>
},
// Other streams, if any
],
"descriptions" : [
// Updated descriptions for the published streams; see "publish" for syntax; optional
]
}
\endverbatim
*
* As you can see, it's basically the same properties as those listed for
* \c publish , with the addition of a \c streams array that can be used
* to tweak individual streams (which is not available when publishing
* since in that case the stream doesn't exist yet). Notice that the
* \c configure request can also be used in renegotiations, to provide
* an updated SDP with changes to the published media. If successful,
* a \c configured event will be sent back as before, formatted like this:
*
\verbatim
{
"videoroom" : "event",
"configured" : "ok"
}
\endverbatim
*
* When configuring the room to request the ssrc-audio-level RTP extension,
* ad-hoc events might be sent to all publishers if \c audiolevel_event is
* set to true. These events will have the following format:
*
\verbatim
{
"videoroom" : <"talking"|"stopped-talking", whether the publisher started or stopped talking>,
"room" : <unique numeric ID of the room the publisher is in>,
"id" : <unique numeric ID of the publisher>,
"audio-level-dBov-avg" : <average value of audio level, 127=muted, 0='too loud'>
}
\endverbatim
*
* An interesting feature VideoRoom publisher can take advantage of is
* RTP forwarding. In fact, while the main purpose of this plugin is
* getting media from WebRTC sources (publishers) and relaying it to
* WebRTC destinations (subscribers), there are actually several use
* cases and scenarios for making this media available to external,
* notnecessarily WebRTC-compliant, components. These components may
* benefit from having access to the RTP media sent by a publisher, e.g.,
* for media processing, external recording, transcoding to other
* technologies via other applications, scalability purposes or
* whatever else makes sense in this context. This is made possible by
* a request called \c rtp_forward which, as the name suggests, simply
* forwards in real-time the media sent by a publisher via RTP (plain
* or encrypted) to a remote backend. Notice that, although we're using
* the term "RTP forwarder", this feature can be used to forward data
* channel messages as well.
*
* You can add a new RTP forwarder for an existing publisher using the
* \c rtp_forward request, which has to be formatted as follows:
*
\verbatim
{
"request" : "rtp_forward",
"room" : <unique numeric ID of the room the publisher is in>,
"publisher_id" : <unique numeric ID of the publisher to relay externally>,
"host" : "<host address to forward the RTP and data packets to>",
"host_family" : "<ipv4|ipv6, if we need to resolve the host address to an IP; by default, whatever we get>",
"streams" : [
{
"mid" : "<mid of publisher stream to forward>",
"host" : "<host address to forward the packets to; optional, will use global one if missing>",
"host_family" : "<optional, will use global one if missing>",
"port" : <port to forward the packets to>,
"ssrc" : <SSRC to use to use when forwarding; optional, and only for RTP streams, not data>,
"pt" : <payload type to use when forwarding; optional, and only for RTP streams, not data>,
"rtcp_port" : <port to contact to receive RTCP feedback from the recipient; optional, and only for RTP streams, not data>,
"simulcast" : <true|false, set to true if the source is simulcast and you want the forwarder to act as a regular viewer (single stream being forwarded) or false otherwise (substreams forwarded separately); optional, default=false>,
"port_2" : <if video and simulcasting, port to forward the packets from the second substream/layer to>,
"ssrc_2" : <if video and simulcasting, SSRC to use to use the second substream/layer; optional>,
"pt_2" : <if video and simulcasting, payload type to use the second substream/layer; optional>,
"port_3" : <if video and simulcasting, port to forward the packets from the third substream/layer to>,
"ssrc_3" : <if video and simulcasting, SSRC to use to use the third substream/layer; optional>,
"pt_3" : <if video and simulcasting, payload type to use the third substream/layer; optional>,
},
{
.. other streams, if needed..
}
],
"srtp_suite" : <length of authentication tag (32 or 80); optional>,
"srtp_crypto" : "<key to use as crypto (base64 encoded key as in SDES); optional>"
}
\endverbatim
*
* As you can see, you basically configure each stream to forward in a
* dedicated object of the \c streams array: for RTP streams (audio, video)
* this includes optionally overriding payload type or SSRC; simulcast
* streams can be forwarded separately for each layer. The only parameters
* you MUST specify are the host and port to send the packets to: the host
* part can be put in the global part of the request, if all streams will
* be sent to the same IP address, while the port must be specific to the
* stream itself.
*
* Notice that, as explained above, in case you configured an \c admin_key
* property and extended it to RTP forwarding as well, you'll need to provide
* it in the request as well or it will be rejected as unauthorized. By
* default no limitation is posed on \c rtp_forward .
*
* It's worth spending some more words on how to forward simulcast publishers,
* as this can lead to some confusion. There are basically two ways to forward
* a simulcast publisher:
*
* -# you treat the forwarder as a regular viewer, which means you still only
* forward a single stream to the recipient, that is the highest quality
* available at any given time: you can do that by setting
* <code>simulcast: true</code> in the \c rtp_forward request;
* -# you forward each substream separately instead, to different target
* ports: you do that by specifying \c video_port_2 , \c video_port_3 and
* optionally the other related \c _2 and \c _3 properties; this is what
* you should use when you want to forward to a simulcast-aware Streaming
* mountpoint (see the \ref streaming for more details).
*
* The two approaches are mutually exclusive: you can NOT use them together
* in the same RTP forwarder.
*
* A successful request will result in an \c rtp_forward response, containing
* the relevant info associated to the new forwarder(s):
*
\verbatim
{
"videoroom" : "rtp_forward",
"room" : <unique numeric ID, same as request>,
"publisher_id" : <unique numeric ID, same as request>,
"forwarders" : [
{
"stream_id" : <unique numeric ID assigned to this forwarder, if any>,
"type" : "<audio|video|data>",
"host" : "<host this forwarder is streaming to, same as request if not resolved>",
"port" : <port this forwarder is streaming to, same as request if configured>,
"local_rtcp_port" : <local port this forwarder is using to get RTCP feedback, if any>,
"remote_rtcp_port" : <remote port this forwarder is getting RTCP feedback from, if any>,
"ssrc" : <SSRC this forwarder is using, same as request if configured>,
"pt" : <payload type this forwarder is using, same as request if configured>,
"substream" : <video substream this video forwarder is relaying, if any>,
"srtp" : <true|false, whether the RTP stream is encrypted (not used for data)>
},
// Other forwarders, if configured
]
}
\endverbatim
*
* To stop a previously created RTP forwarder and stop it, you can use
* the \c stop_rtp_forward request, which has to be formatted as follows:
*
\verbatim
{
"request" : "stop_rtp_forward",
"room" : <unique numeric ID of the room the publisher is in>,
"publisher_id" : <unique numeric ID of the publisher to update>,
"stream_id" : <unique numeric ID of the RTP forwarder>
}
\endverbatim
*
* A successful request will result in a \c stop_rtp_forward response:
*
\verbatim
{
"videoroom" : "stop_rtp_forward",
"room" : <unique numeric ID, same as request>,
"publisher_id" : <unique numeric ID, same as request>,
"stream_id" : <unique numeric ID, same as request>
}
\endverbatim
*
* To get a list of all the forwarders in a specific room, instead, you
* can make use of the \c listforwarders request, which has to be
* formatted as follows:
*
\verbatim
{
"request" : "listforwarders",
"room" : <unique numeric ID of the room>,
"secret" : "<room secret; mandatory if configured>"
}
\endverbatim
*
* A successful request will produce a list of RTP forwarders in a
* \c forwarders response:
*
\verbatim
{
"videoroom" : "forwarders",
"room" : <unique numeric ID of the room>,
"publishers" : [ // Array of publishers with RTP forwarders
{ // Publisher #1
"publisher_id" : <unique numeric ID of publisher #1>,
"forwarders" : [ // Array of RTP forwarders
{ // RTP forwarder #1
"stream_id" : <unique numeric ID assigned to this RTP forwarder, if any>,
"type" : "<audio|video|data>",
"host" : "<host this forwarder is streaming to>",
"port" : <port this forwarder is streaming to>,
"local_rtcp_port" : <local port this forwarder is using to get RTCP feedback, if any>,
"remote_rtcp_port" : <remote port this forwarder getting RTCP feedback from, if any>,
"ssrc" : <SSRC this forwarder is using, if any>,
"pt" : <payload type this forwarder is using, if any>,
"substream" : <video substream this video forwarder is relaying, if any>,
"srtp" : <true|false, whether the RTP stream is encrypted>
},
// Other forwarders for this publisher
],
},
// Other publishers
]
}
\endverbatim
*
* To enable or disable recording on all participants while the conference
* is in progress, you can make use of the \c enable_recording request,
* which has to be formatted as follows:
*
\verbatim
{
"request" : "enable_recording",
"room" : <unique numeric ID of the room>,
"secret" : "<room secret; mandatory if configured>"
"record" : <true|false, whether participants in this room should be automatically recorded or not>,
}
\endverbatim
*
* Notice that, as we'll see later, participants can normally change their
* own recording state via \c configure requests as well: this was done to
* allow the maximum flexibility, where rather than globally or automatically
* record something, you may want to individually record some streams and
* to a specific file. That said, if you'd rather ensure that participants
* can't stop their recording if a global recording is enabled, or start
* it when the room is not supposed to be recorded instead, then you should
* make sure the room is created with the \c lock_record property set to
* \c true : this way, the recording state can only be changed if the room
* secret is provided, thus ensuring that only an administrator will normally
* be able to do that (e.g., using the \c enable_recording just introduced).
*
* To conclude, you can leave a room you previously joined as publisher
* using the \c leave request. This will also implicitly unpublish you
* if you were an active publisher in the room. The \c leave request
* looks like follows:
*
\verbatim
{
"request" : "leave"
}
\endverbatim
*
* If successful, the response will look like this: