-
Notifications
You must be signed in to change notification settings - Fork 2.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ICE crashes #481
Comments
Looks like a problem encrypting an SRTP packet. Not sure if this is happening because the context is not valid anymore (handle removed or PeerConnection closed) and yet a plugin is still pushing data, or if some data somewhere got corrupted. Compiling with AddressSanitizer support may provide some more info: https://janus.conf.meetecho.com/docs/debug |
@lminiero , this is a MT issue, I also encountered the same problem。 In janus, it may be start two ice_send_thread 。fix with following code. if(state == NICE_COMPONENT_STATE_CONNECTED || state == NICE_COMPONENT_STATE_READY) {
/* NOTICE: can't use handle->mutex in this function, janus_process_incoming_request --> janus_ice_trickle_parse locked it */
if(!g_atomic_int_compare_and_exchange(&handle->send_thread_creating, 0, 1)) {
return;
}
if(handle->send_thread == NULL) {
/* Start the outgoing data thread */
GError *error = NULL;
handle->send_thread = g_thread_try_new("ice send thread", &janus_ice_send_thread, handle, &error);
if(error != NULL) {
/* FIXME We should clear some resources... */
JANUS_LOG(LOG_ERR, "[%"SCNu64"] Got error %d (%s) trying to launch the ICE send thread...\n", handle->handle_id, error->code, error->message ? error->message : "??");
janus_mutex_unlock(&handle->mutex);
}
}
g_atomic_int_set(&handle->send_thread_creating, 0);
return;
} |
Not sure what you want me to fix? |
Sorry, just noticed YOU added |
first time I use handle->mutex to lock it, but that will cause a dead lock. this callback will be called in ice event loop and nice_agent_set_remote_candidates . |
But have you actually noticed the thread was being created twice, or was it a guess? Just wondering what may lead to this behaviour: it shouldn't be possible for the thread to start twice, as we have a check in place already ( |
I find the answer from the log,
|
I think we might be running into the same problem again. Getting another Segfault in Ice. #0 0x00007f6950bcea0a in EVP_DigestFinal_ex () from /usr/lib64/libcrypto.so.1.0.0 |
I'll integrate the suggestion @jing3018 made tomorrow, and we'll see if it fixes it for you. |
@lminiero using libnice-0.1.13
|
The Knowing that this can happen, I'll work on adding the fix you suggested, so that we're really sure the thread is only created once. |
@YtelDrew @RanceYtel @jing3018 just added the fix, let me know if this solves it for you and I'll close the issue. |
i pulled and updated the latest, its currenting running now. i'll keep you posted on how it goes. |
no crash on the server that was crashy pre-fix so it appears to solve the problem. |
Cool, thanks for the feedback! Kudos to @jing3018 for finding a fix so quickly. |
I've updated from a few revisions ago, and now i'm experiencing this crash
[2556358.156303] ice send thread[21831]: segfault at 110 ip 0000000000000110 sp 00007fa76fffcbb8 error 14 in janus[400000+69000]
Thread 1 (Thread 0x7fa76ffff700 (LWP 21831)):
#0 0x0000000000000110 in ?? ()
#1 0x00007fa8947482ad in CRYPTO_ctr128_encrypt_ctr32 () from /usr/lib64/libcrypto.so.1.0.0
#2 0x00007fa894798390 in ?? () from /usr/lib64/libcrypto.so.1.0.0
#3 0x00007fa89479407b in EVP_EncryptUpdate () from /usr/lib64/libcrypto.so.1.0.0
#4 0x00007fa8939f58f2 in aes_icm_openssl_encrypt () from /usr/lib64/libsrtp.so.1
#5 0x00007fa8939f3378 in srtp_protect () from /usr/lib64/libsrtp.so.1
#6 0x00000000004226b6 in janus_ice_send_thread ()
#7 0x00007fa894f6cf85 in ?? () from /usr/lib64/libglib-2.0.so.0
#8 0x00007fa8937d93a4 in start_thread () from /lib64/libpthread.so.0
#9 0x00007fa89351c18d in clone () from /lib64/libc.so.6
i'm going to wipe it out and try again tomorrow with debugging enabled to see if i can find the source of this crash.
thanks
The text was updated successfully, but these errors were encountered: