-
Notifications
You must be signed in to change notification settings - Fork 2.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Segmentation fault in multistream Videoroom on handle detach #2775
Comments
Trace doesn't say much (especially without symbols), a libasan trace would be better: https://janus.conf.meetecho.com/docs/debug.html |
Gotcha, we'll get libasan in our build and post an update if the crash happens again. |
@OxleyS can you apply this tentative patch and see if it fixes the issue for you?
|
I tried applying the patch, but it doesn't seem to resolve the crashes. We've managed to put together this tool that can recreate the crash semi-consistently (I wish it was easier to share!), so I'm going to play around with that for a while and see if I can narrow down the conditions it crashes under. |
@OxleyS are the stack traces identical to the previous reports ? |
The conditions for the crash are clear to us: a publisher and one of his subscribers leaving at the same time. |
Having access to a tool that consistently replicates the issue would help, though. Just relying on contributed dumps can only help up to a certain extent, since we'd have no way to check if our attempts fix things or not. |
Yeah, stack traces look identical. As for the tool, in its current form it's pretty heavily tied to our internal environment, but we can take a shot at porting it to something more shareable. |
Here's a setup for reproduction https://github.com/thatsmydoing/janus-gateway-2775 it's not exactly the same as what we used to get the previous stack traces but it's close enough? I can at least trigger it on d93290f + the provided patch. I've seen it usually takes between 100-500 attempts to trigger but sometimes longer |
thanks @thatsmydoing , we managed to reproduce this bug (and many others too...) |
With the attached patch Janus successfully completed 3000+ iterations, whereas it usually crashes in less than 100 on my env. @OxleyS can you please give it a try to confirm that it fixes the issue? As a side note we observed some memory leaks. Those should not be regressions since they seem to happen on vanilla |
We've tried the attached patch and haven't gotten any crashes over 20k+ iterations, so I think this fixes it! |
I guess we can close this then. Thanks for the feedback! |
The mentioned leaks have now been fixed in 624df49 |
This was on the
multistream
branch, commit06c18221ba73aa609d2c7f88ffc3a06fddb2b52e
.This crash was triggered when users left the room. I don't know whether it was the publisher side or subscriber side specifically that triggered this. Here's a GDB stacktrace:
My first thought was that the stacktrace implies a potential race with the Janus session's destruction, but my application logs don't indicate we were trying to destroy the session at the time. All Janus handles are managed under the same Janus session in our application.
Logs just before the crash seem pretty unassuming but I'll post them here as well:
The text was updated successfully, but these errors were encountered: