You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I think one of our janus servers experienced a deadlock in the streaming plugin. The websocket API became unresponsive and I was not able to create/destroy mountpoint anymore (no response from the streaming plugin API)
I think the deadlock happens on the mountpoints_mutex mutex.
To determine that, I use the admin API to communicate with the streaming plugin.
If sending :
{
"request": "info"
}
I receive the reply a reply from the streaming plugin :
But, if trying to request info on a non-existing endpoint :
{
"request": "info",
"id": "123"
}
In that case the request never completes, and no log messages are shown (I was expecting No such mountpoint/stream), so I guess the query is stuck here :
I wasn't able to reproduce the issue yet, but I'll update the ticket when I have some news.
Notes:
this was on 9eeeb38 (3 commits from master as of today)
not sure if that's a coincidence, but if looks like this happened few seconds after 6-7 mountpoint destroy were done in a short period of time
existing streaming plugin RTSP mountpoints are still running (I can see them looping on RTSP retries because the source is gone, but I can't destroy those mountpoints)
the videoroom plugin is also running on the same server, and continue to work even if the streaming plugin API seems dead
The text was updated successfully, but these errors were encountered:
If you want to debug locks, you can use our lock debugging feature, which will print any attempt to lock/unlock and the file/line where it happened. Ideally you'll want it to start from the beginning (enabled in janus.jcfg), but it can be enabled/disabled dynamically via Admin API as well. Of course, you should ensure it's enabled before the issue starts, or you may be missing info. Notice it might also greatly increase the log size.
I think one of our janus servers experienced a deadlock in the streaming plugin. The websocket API became unresponsive and I was not able to create/destroy mountpoint anymore (no response from the streaming plugin API)
I think the deadlock happens on the
mountpoints_mutex
mutex.To determine that, I use the admin API to communicate with the streaming plugin.
If sending :
I receive the reply a reply from the streaming plugin :
Which is handled by
janus-gateway/plugins/janus_streaming.c
Lines 2489 to 2499 in 2d83e96
But, if trying to request info on a non-existing endpoint :
In that case the request never completes, and no log messages are shown (I was expecting
No such mountpoint/stream
), so I guess the query is stuck here :janus-gateway/plugins/janus_streaming.c
Line 2510 in 2d83e96
I wasn't able to reproduce the issue yet, but I'll update the ticket when I have some news.
Notes:
The text was updated successfully, but these errors were encountered: