dead lock

[mightZhong]

Hi , we find there is a dead lock in websocket client gone scenario.
As we use an old version janus source code, so the code line in dump information may not correspond to the lastest code.

In general, thread janus_websockets_thread locked client->mutex, wait sessions_mutex, meanwhile janus_sessions_watchdog locked sessions_mutex, wait client->mutex, below is the detal information.

Would you please have a look, thanks

#3  0x00007efc085c3a3d in janus_websockets_send_message (transport=, request_id=,
    admin=, message=0x7efbf4206e60) at transports/janus_websockets.c:867
867	transports/janus_websockets.c: No such file or directory.
(gdb) p client->mutex
$1 = {__data = {__lock = 2, __count = 0, __owner = 9944, __nusers = 1, __kind = 0, __spins = 0, __elision = 0,
    __list = {__prev = 0x0, __next = 0x0}},
  __size = "\002\000\000\000\000\000\000\000\330&\000\000\001", '\000' , __align = 2}
(gdb) thread 31
[Switching to thread 31 (Thread 0x7efbfb7fe700 (LWP 9944))]
#0  0x00007efc40ab054d in __lll_lock_wait () from /lib64/libpthread.so.0
(gdb) bt
#0  0x00007efc40ab054d in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007efc40aabe9b in _L_lock_883 () from /lib64/libpthread.so.0
#2  0x00007efc40aabd68 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00000000004584db in janus_transport_gone (plugin=, transport=0x7efbe03dc6c0)
    at janus.c:2200
#4  0x00007efc085bcc0b in janus_websockets_destroy_client (ws_client=ws_client@entry=0x7efbe0373800,
    wsi=wsi@entry=0x7efbe0093f80, log_prefix=log_prefix@entry=0x7efc085c41a8 "WSS")
    at transports/janus_websockets.c:789
#5  0x00007efc085bd018 in janus_websockets_common_callback (wsi=0x7efbe0093f80, reason=,
    user=0x7efbe0373800, in=, len=, admin=)
    at transports/janus_websockets.c:1243
#6  0x00007efc0837ed0b in __lws_close_free_wsi () from /opt/dep-libs/lib/libwebsockets.so.15
#7  0x00007efc083825e0 in lws_service_fd_tsi () from /opt/dep-libs/lib/libwebsockets.so.15
#8  0x00007efc083a0047 in _lws_plat_service_tsi () from /opt/dep-libs/lib/libwebsockets.so.15
#9  0x00007efc0838271d in lws_service () from /opt/dep-libs/lib/libwebsockets.so.15
#10 0x00007efc085c03ff in janus_websockets_thread (data=0x1cf13c0) at transports/janus_websockets.c:937
#11 0x00007efc41bfb255 in g_thread_proxy (data=0x1be64a0) at gthread.c:780
#12 0x00007efc40aa9ea5 in start_thread () from /lib64/libpthread.so.0
#13 0x00007efc3f6a68dd in clone () from /lib64/libc.so.6



(gdb) thread 35
[Switching to thread 35 (Thread 0x7efbf915c700 (LWP 9948))]
#0  0x00007efc40ab054d in __lll_lock_wait () from /lib64/libpthread.so.0
(gdb) bt
#0  0x00007efc40ab054d in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007efc40aabe9b in _L_lock_883 () from /lib64/libpthread.so.0
#2  0x00007efc40aabd68 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00007efc085c3a3d in janus_websockets_send_message (transport=, request_id=,
    admin=, message=0x7efbf4206e60) at transports/janus_websockets.c:867
#4  0x000000000045c6fd in janus_session_notify_event (session=session@entry=0x7efc242cd9e0,
    event=event@entry=0x7efbf4206e60) at janus.c:458
#5  0x000000000046700a in janus_plugin_push_event (plugin_session=, plugin=,
    transaction=, message=, jsep=) at janus.c:2337
#6  0x00007efc38267d6a in saturn_videoroom_notify_common (participant=0x7efc04195e60, msg=0x7efbf419dc80,
    type=0x7efc382ae339 "LEAVING") at plugins/saturn_videoroom.c:1444
#7  0x00007efc382705ac in saturn_videoroom_notify_leave (publisher=publisher@entry=0x7efc044145f0)
    at plugins/saturn_videoroom.c:1500
#8  0x00007efc38272188 in saturn_videoroom_hangup_media (handle=handle@entry=0x7efbd44e8cc0)
    at plugins/saturn_videoroom.c:4746
#9  0x00007efc382726f3 in saturn_videoroom_destroy_session (handle=0x7efbd44e8cc0, error=0x7efbf915b654)
    at plugins/saturn_videoroom.c:3674
#10 0x00000000004489c7 in janus_ice_handle_destroy (session=session@entry=0x7efbd40b8c40, handle=0x7efbd43ef0c0)
    at ice.c:977
#11 0x000000000045ccb7 in janus_session_handles_clear (session=session@entry=0x7efbd40b8c40) at janus.c:521
#12 0x000000000045cd98 in janus_session_destroy (session=session@entry=0x7efbd40b8c40) at janus.c:472
#13 0x000000000045d14d in janus_check_sessions (
    user_data=) at janus.c:383
#14 0x00007efc41bd7513 in g_timeout_dispatch (source=0x7efbf40008c0, callback=,
    user_data=) at gmain.c:4577
#15 0x00007efc41bd6ae5 in g_main_dispatch (context=0x1c3f070) at gmain.c:3154
#16 g_main_context_dispatch (context=context@entry=0x1c3f070) at gmain.c:3769
#17 0x00007efc41bd6e58 in g_main_context_iterate (context=0x1c3f070, block=block@entry=1,
    dispatch=dispatch@entry=1, self=) at gmain.c:3840
#18 0x00007efc41bd711a in g_main_loop_run (loop=0x1d01ef0) at gmain.c:4034
#19 0x0000000000459b25 in janus_sessions_watchdog (user_data=0x1d01ef0) at janus.c:404
#20 0x00007efc41bfb255 in g_thread_proxy (data=0x1be64f0) at gthread.c:780
#21 0x00007efc40aa9ea5 in start_thread () from /lib64/libpthread.so.0
#22 0x00007efc3f6a68dd in clone () from /lib64/libc.so.6

[lminiero]

@mightZhong please pay some attention when you create issues... you had left the placeholder text there, and had just copied the gdb trace inline which messed up issues references. I fixed this for you this time, next time I'll just close the issue.

[lminiero]

I see you're using custom code in there (saturn videoroom something). I'm afraid we can't debug something that's not related to an unmodified Janus here, so I'll have to close this. If you can replicate with any of the existing plugins, feel free to reopen.

[mightZhong]

I read the lastest code, the dead lock issue is also exist.

[lminiero]

You still need to provide a way to replicate this with stock plugins, not yours, otherwise we can't look at it.