Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Auto switch simulcast layer if REMB is too low #2735

Closed
wants to merge 4 commits into from
Closed

Auto switch simulcast layer if REMB is too low #2735

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
562ba78
POC: automatically switch to lowest simulcast layer after receiving R…
sjkummer Jul 13, 2021
e2302e8
introduce min_rec_bitrate_simulcast setting in videoroom for simulcas…
sjkummer Jul 13, 2021
4166d2f
typos
sjkummer Jul 13, 2021
192dd4f
default simulcast_min_bitrate to 256000
sjkummer Jul 13, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions conf/janus.plugin.videoroom.jcfg.sample
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,13 @@ general: {
# By default, integers are used as a unique ID for both rooms and participants.
# In case you want to use strings instead (e.g., a UUID), set string_ids to true.
#string_ids = true

#min_rec_bitrate_simulcast=0 # The minimum bitrate required to subscribe higher
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't you need two values here, one as the minimum bitrate for the low spatial layer and a second one for the mid one?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also see at least two but i would say all three values might even make more sense.
If we do not reach the lowest bitrate janus could also drop the fps (but i would make that in a second task)
Having the bitrates on hand alows further improvement in the future.

e.g.
min_required_bitrate_vp8_sl_0: 150000
min_required_bitrate_vp8_sl_1: 500000
min_required_bitrate_vp8_sl_2: 1200000

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds overly specific: I don't want anything codec specific in there. Besides, I don't think it's a good idea to have generic room settings here, especially if the idea is to match the simulcast targets of publishers, who may be configured differently even when in the same room.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The question is how do you want janus to select the proper layer on the subscriber side if the plugin handle is not aware about the publisher configured bitrates for the layers? You don´t have them in janus as the publisher client has set them on the client side. (I also don´t see them in the general settings, i see them in an api call on the subscriber side)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What sounds more interesting is tracking the bitrate for the different substreams for all publishers, and use those as triggers, which would make it more dynamic as well (even though more complex, as we'd have to make sure substreams that are just starting don't confuse the logic, e.g., because the high substream temporarily results with a lower bitrate than the medium/low one).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don´t have them in janus as the publisher client has set them on the client side

We never needed to know them. Having the publisher optionally pass the settings they used via API sounds viable too, even though not necessarily reliable, since those are targets and not the actual bitrate in use.

Copy link
Contributor

@JanFellner JanFellner Jul 13, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We never needed to know them.

For the publisher side yeah, currently the browser is handling the layer sending according to available bitrates so no need to know them inside janus. For the subscriber side i think its the easist to start with an api that allows to set them dynamic and see if it works in real live (but yes you are totally right - having them in sync with the publisher side would make the most sense but is from the current perspective way more effort)
We will sync tomorrow @fippo - @sjkummer and me (you are always invited :D)

# simulcast layers. If REMB is lower than that,
# the subscriber will automatically be switched
# to the lowest simulcast layer (default=256000).
# This can avoid creating congestion for simulcast
# subscribers.
}

room-1234: {
Expand Down
39 changes: 39 additions & 0 deletions plugins/janus_videoroom.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1502,6 +1502,7 @@ static GHashTable *rooms;
static janus_mutex rooms_mutex = JANUS_MUTEX_INITIALIZER;
static char *admin_key = NULL;
static gboolean lock_rtpfwd = FALSE;
static int simulcast_min_bitrate = 256000;

typedef struct janus_videoroom_session {
janus_plugin_session *handle;
Expand Down Expand Up @@ -2214,6 +2215,9 @@ int janus_videoroom_init(janus_callbacks *callback, const char *config_path) {
if(string_ids) {
JANUS_LOG(LOG_INFO, "VideoRoom will use alphanumeric IDs, not numeric\n");
}
janus_config_item *simulcast_min_bitrate_config = janus_config_get(config, config_general, janus_config_type_item, "min_rec_bitrate_simulcast");
if(simulcast_min_bitrate_config != NULL && simulcast_min_bitrate_config->value != NULL)
simulcast_min_bitrate = atol(simulcast_min_bitrate_config->value);
}
rooms = g_hash_table_new_full(string_ids ? g_str_hash : g_int64_hash, string_ids ? g_str_equal : g_int64_equal,
(GDestroyNotify)g_free, (GDestroyNotify)janus_videoroom_room_destroy);
Expand Down Expand Up @@ -5515,6 +5519,41 @@ void janus_videoroom_incoming_rtcp(janus_plugin_session *handle, janus_plugin_rt
uint32_t bitrate = janus_rtcp_get_remb(buf, len);
if(bitrate > 0) {
/* FIXME We got a REMB from this subscriber, should we do something about it? */
// JANUS_LOG(LOG_VERB, "Received REMB: %d\n", bitrate);

if (simulcast_min_bitrate > 0) {
if (bitrate < simulcast_min_bitrate) {

const target_layer = 0;

janus_mutex_lock(&session->mutex);
janus_videoroom_publisher *publisher = s->feed;
/* It is safe to use feed as the only other place sets feed to NULL
is in this function and accessing to this function is synchronized
by sessions_mutex */

if((publisher->ssrc[0] != 0 || publisher->rid[0] != NULL)) {

if (s->sim_context.substream_target != target_layer) {
JANUS_LOG(LOG_VERB, "Switching to lowest simulcast layer (0) because received REMB (%d) was lower than the defined minimum (%d)\n", bitrate, simulcast_min_bitrate);
s->sim_context.substream_target = target_layer;
if(s->sim_context.substream_target >= 0 && s->sim_context.substream_target <= 2) {
JANUS_LOG(LOG_VERB, "Setting video SSRC to let through (simulcast): %"SCNu32" (index %d, was %d)\n",
publisher->ssrc[s->sim_context.substream_target],
s->sim_context.substream_target,
s->sim_context.substream);
}
if(s->sim_context.substream_target == s->sim_context.substream) {
/* No need to do anything, we're already getting the right substream, so notify the user */
} else {
/* Send a FIR */
janus_videoroom_reqpli(publisher, "Simulcasting substream change");
}
}
}
janus_mutex_unlock(&session->mutex);
}
}
}
janus_refcount_decrease_nodebug(&s->ref);
}
Expand Down