Crash when streaming data over UDP #3672

romain145 · 2024-01-11T11:40:26Z

What happened?

When using audio-reactive-led-strip to send LED data over UDP the ESP32 reboots after about 10sec of streaming with the following message:

Guru Meditation Error: Core  1 panic'ed (LoadStoreError). Exception was unhandled.
Core 1 register dump:
PC      : 0x4010f783  PS      : 0x00060c30  A0      : 0x8011450f  A1      : 0x3ffb1e60
A2      : 0xffffffff  A3      : 0x3ffb1e70  A4      : 0x3ffffffe  A5      : 0x0004e18e  
A6      : 0x3ffc7ecc  A7      : 0x3ffb1e70  A8      : 0x8010f798  A9      : 0x3ffb1e40  
A10     : 0x00000000  A11     : 0x00000000  A12     : 0x00000000  A13     : 0x00000000
A14     : 0x00000000  A15     : 0x00000000  SAR     : 0x0000000a  EXCCAUSE: 0x00000003  
EXCVADDR: 0x40000001  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xfffffffe

ELF file SHA256: 0000000000000000

Backtrace: 0x4010f780:0x3ffb1e60 0x4011450c:0x3ffb1f50 0x40114f1e:0x3ffb1f90 0x401338ad:0x3ffb1fb0 0x4008b89e:0x3ffb1fd0
  #0  0x4010f780:0x3ffb1e60 in handleNotifications() at wled00/udp.cpp:530 (discriminator 1)
  #1  0x4011450c:0x3ffb1f50 in WLED::loop() at wled00/wled.cpp:62
  #2  0x40114f1e:0x3ffb1f90 in loop() at Z:/git/github/WLED/wled00/wled00.ino:20
  #3  0x401338ad:0x3ffb1fb0 in loopTask(void*) at C:\Users\romai\.platformio\packages\framework-arduinoespressif32\cores\esp32/main.cpp:23
  #4  0x4008b89e:0x3ffb1fd0 in vPortTaskWrapper at /home/cschwinne/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

Rebooting...

Running 75e857a with the following PIO config as suggested here #3637 (comment)

extends = env:esp32dev
monitor_filters = esp32_exception_decoder 
build_flags = ${common.build_flags_esp32} ${common.debug_flags} -D WLED_RELEASE_NAME=ESP32_debug

To Reproduce Bug

Run visualisation.py from https://github.com/scottlawsonbc/audio-reactive-led-strip
Update config.py with the correct IP and UDP port.

Expected Behavior

Should run without crashing.

Install Method

Self-Compiled

What version of WLED?

WLED 0.14.1-b3 (build 2401060)

Which microcontroller/board are you seeing the problem on?

ESP32

Relevant log/trace output

No response

Anything else?

May be related to #3637

Code of Conduct

I agree to follow this project's Code of Conduct

The text was updated successfully, but these errors were encountered:

blazoncek · 2024-01-11T12:25:09Z

Can you confirm (by adding DEBUG_PRINTs) that the line is 530? It is illogical as it is a for loop statement.
It would also be beneficial if you can capture a packet that causes the exception.

blazoncek · 2024-01-11T12:36:54Z

And please provide WLED configuration (cfg.json).

blazoncek · 2024-01-11T12:50:09Z

Tracing the changes, this code has not been changed in about 2 years. And only changed uint16_t to size_t.

romain145 · 2024-01-11T13:59:15Z

Line 530 before the for loop I added this to monitor the packetSize:
DEBUG_PRINTLN("WARLS packetSize: " + String(packetSize));

This is the trace:

Connected! IP address: 192.168.1.37
Init STA interfaces
mDNS started
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 114
WARLS packetSize: 98
WARLS packetSize: 106
WARLS packetSize: 98
WARLS packetSize: 122
WARLS packetSize: 98
WARLS packetSize: 98
WARLS packetSize: 106
WARLS packetSize: 106
WARLS packetSize: 82
WARLS packetSize: 106
WARLS packetSize: 98
WARLS packetSize: 90
WARLS packetSize: 98
WARLS packetSize: 90
WARLS packetSize: 82
WARLS packetSize: 98
WARLS packetSize: 74
WARLS packetSize: 82
WARLS packetSize: 98
WARLS packetSize: 82
WARLS packetSize: 90
WARLS packetSize: 90
WARLS packetSize: 82
WARLS packetSize: 66
WARLS packetSize: 66
WARLS packetSize: 58
WARLS packetSize: 114
WARLS packetSize: 106
WARLS packetSize: 82
WARLS packetSize: 82
WARLS packetSize: 114
WARLS packetSize: 18
WARLS packetSize: 74
WARLS packetSize: 98
WARLS packetSize: 90
WARLS packetSize: 82
WARLS packetSize: 90
WARLS packetSize: 90
WARLS packetSize: 90
WARLS packetSize: 122
WARLS packetSize: 122
WARLS packetSize: 106
WARLS packetSize: 122
WARLS packetSize: 18
WARLS packetSize: 2
Guru Meditation Error: Core  1 panic'ed (LoadStoreError). Exception was unhandled.
Core 1 register dump:
PC      : 0x40112094  PS      : 0x00060c30  A0      : 0x8011447f  A1      : 0x3ffb1e60  
A2      : 0xffffffff  A3      : 0x3ffb1e70  A4      : 0x3ffffffe  A5      : 0x0004e18e
A6      : 0x3ffc7ecc  A7      : 0x3ffb1e70  A8      : 0x801120a8  A9      : 0x3ffb1e40  
A10     : 0x00000000  A11     : 0x00000000  A12     : 0x00000000  A13     : 0x00000000
A14     : 0x00000000  A15     : 0x00000000  SAR     : 0x00000010  EXCCAUSE: 0x00000003  
EXCVADDR: 0x40000001  LBEG    : 0x4000c28c  LEND    : 0x4000c296  LCOUNT  : 0x00000000

ELF file SHA256: 0000000000000000

Backtrace: 0x40112091:0x3ffb1e60 0x4011447c:0x3ffb1f50 0x40114e8e:0x3ffb1f90 0x4013381d:0x3ffb1fb0 0x4008b89e:0x3ffb1fd0
  #0  0x40112091:0x3ffb1e60 in handleNotifications() at wled00/udp.cpp:531 (discriminator 1)
  #1  0x4011447c:0x3ffb1f50 in WLED::loop() at wled00/wled.cpp:62
  #2  0x40114e8e:0x3ffb1f90 in loop() at Z:/git/github/WLED/wled00/wled00.ino:20
  #3  0x4013381d:0x3ffb1fb0 in loopTask(void*) at C:\Users\romai\.platformio\packages\framework-arduinoespressif32\cores\esp32/main.cpp:23
  #4  0x4008b89e:0x3ffb1fd0 in vPortTaskWrapper at /home/cschwinne/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

Rebooting...

There is an underflow when packetSize == 2 line 530 in
for (size_t i = 2; i < packetSize -3; i += 4)

Moreover I'm under the impression that totalLen = strip.getLengthTotal(); should terminate the loop if the number of packets received is too large?

Here is the wled_cfg.json

softhack007 · 2024-01-11T14:19:33Z

There is an underflow when packetSize == 2 line 530 in
for (size_t i = 2; i < packetSize -3; i += 4)

Indeed the part "packetsize -3" may underflow, as everything is computed as unsigned, and "2U - 3U = UINT32_MAX".

Please add the following right before the "for(" line: if (packetSize > 2), and check if it works better.

romain145 · 2024-01-11T14:45:15Z

Yes I made the modification and the crash is gone.
I'll investigate audio-reactive-led-strip why small packets are sent.
Thanks

when only two bytes were received in a packet, the "for" condition `packetsize -3` would underflow, leading to an infinite loop.

blazoncek · 2024-01-11T17:13:44Z

The problem is packetSize == 2 as, if you look at source more closely, there is if (packetSize < 2) return; at the start of the WARLS code. This line should read if (packetSize < 5) return; instead.

blazoncek · 2024-01-11T17:14:48Z

Yes I made the modification and the crash is gone. I'll investigate audio-reactive-led-strip why small packets are sent. Thanks

There may be a bug in there as 2-byte packet only contains header and timeout seconds. No LED data.

softhack007 · 2024-01-11T17:20:20Z

This line should read if (packetSize < 5) return; instead.

I was not sure if other formats (drgb) allow for 2-bytes packets. What is the minimal size for a "correct" UDP realtime packet?
We could then reject anything that is "too small" or "too large".

blazoncek · 2024-01-11T17:23:14Z

Yes, you are correct. on more thorough inspection there are other possible valid packets <5.

blazoncek · 2024-01-11T17:25:58Z

warls: 6 byte minimum
drgb: 5 byte minimum
drgbw: 6 byte minimum
dnrgbw: 7 byte minimum

that is assuming at least 1 LED data is present

blazoncek · 2024-01-11T17:30:01Z

perhaps the easiest solution is to change size_t i into int i.

softhack007 · 2024-01-11T21:54:53Z

perhaps the easiest solution is to change size_t i into int i.

May work, but i'm not 100% sure how gcc will handle the resulting comparison between signed and unsigned.
Maybe its better to make assumptions explicit as you did in ea32c97.

Another question is, why did WLED receive short packages? Could it be fragmentation of large network packets? Or just a coding bug in the tool that sends the data?

Xploder · 2024-01-15T09:29:38Z

Same crash with HyperHDR 20 as it also uses the reactive API now. Will try with final Release 14.1 now.

blazoncek · 2024-01-15T09:34:36Z

The exact case from OP has been resolved in 0.14.1.

Xploder · 2024-01-15T09:47:56Z

Can't confirm, same crashy behaviour with 0.14.1 using ESP32-Ethernet (on WT32-ETH01) and current HyperHDR master.
Rolling back to 0.14.0 fixes the issue.

blazoncek · 2024-01-15T10:29:45Z

@Xploder please do as OP by providing relevant traces and open a new issue.

softhack007 · 2024-01-15T15:55:24Z

Can't confirm, same crashy behaviour with 0.14.1 using ESP32-Ethernet (on WT32-ETH01) and current HyperHDR master.
Rolling back to 0.14.0 fixes the issue.

@Xploder it might look similar, however the root cause of the specific crash that was reported by @romain145 is definitely fixed in 0.14.1.

Please open a new bug report ("New Issue") for the crash you see with 0.14.1.
It would be helpful if you also provide the following with your report:

cfg.json and presets.json (downloaded from your device)
if possible: an example network packet that gets send to WLED by your software (developers usually don't have HyperHDR)
WLED Serial Monitor output, including stack trace (exception decoder) of the crash. See Release Page, heading "How to get a stack dump".

…oookie#3672) when only two bytes were received in a packet, the "for" condition `packetsize -3` would underflow, leading to an infinite loop.

romain145 added the bug label Jan 11, 2024

softhack007 added a commit that referenced this issue Jan 11, 2024

udp realtime - improve robustness against short packets (fix for #3672)

a9eb7cb

when only two bytes were received in a packet, the "for" condition `packetsize -3` would underflow, leading to an infinite loop.

blazoncek added a commit that referenced this issue Jan 11, 2024

Fix for #3672

ea32c97

blazoncek added the fixed in source This issue is unsolved in the latest release but fixed in master label Jan 12, 2024

blazoncek added this to the 0.14.1 candidate milestone Jan 12, 2024

blazoncek closed this as completed Jan 15, 2024

softhack007 pushed a commit to MoonModules/WLED that referenced this issue Jan 15, 2024

Fix for Aircoookie#3672

854425e

blazoncek mentioned this issue Jan 18, 2024

14.1 can't connect to router on 14.1 on ESP8266 Wemos D1 mini #3690

Open

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Crash when streaming data over UDP #3672

Crash when streaming data over UDP #3672

romain145 commented Jan 11, 2024 •

edited

Loading

blazoncek commented Jan 11, 2024

blazoncek commented Jan 11, 2024

blazoncek commented Jan 11, 2024

romain145 commented Jan 11, 2024

softhack007 commented Jan 11, 2024

romain145 commented Jan 11, 2024

blazoncek commented Jan 11, 2024 •

edited

Loading

blazoncek commented Jan 11, 2024

softhack007 commented Jan 11, 2024 •

edited

Loading

blazoncek commented Jan 11, 2024

blazoncek commented Jan 11, 2024

blazoncek commented Jan 11, 2024

softhack007 commented Jan 11, 2024

Xploder commented Jan 15, 2024 •

edited

Loading

blazoncek commented Jan 15, 2024

Xploder commented Jan 15, 2024

blazoncek commented Jan 15, 2024

softhack007 commented Jan 15, 2024 •

edited

Loading

Crash when streaming data over UDP #3672

Crash when streaming data over UDP #3672

Comments

romain145 commented Jan 11, 2024 • edited Loading

What happened?

To Reproduce Bug

Expected Behavior

Install Method

What version of WLED?

Which microcontroller/board are you seeing the problem on?

Relevant log/trace output

Anything else?

Code of Conduct

blazoncek commented Jan 11, 2024

blazoncek commented Jan 11, 2024

blazoncek commented Jan 11, 2024

romain145 commented Jan 11, 2024

softhack007 commented Jan 11, 2024

romain145 commented Jan 11, 2024

blazoncek commented Jan 11, 2024 • edited Loading

blazoncek commented Jan 11, 2024

softhack007 commented Jan 11, 2024 • edited Loading

blazoncek commented Jan 11, 2024

blazoncek commented Jan 11, 2024

blazoncek commented Jan 11, 2024

softhack007 commented Jan 11, 2024

Xploder commented Jan 15, 2024 • edited Loading

blazoncek commented Jan 15, 2024

Xploder commented Jan 15, 2024

blazoncek commented Jan 15, 2024

softhack007 commented Jan 15, 2024 • edited Loading

romain145 commented Jan 11, 2024 •

edited

Loading

blazoncek commented Jan 11, 2024 •

edited

Loading

softhack007 commented Jan 11, 2024 •

edited

Loading

Xploder commented Jan 15, 2024 •

edited

Loading

softhack007 commented Jan 15, 2024 •

edited

Loading