streamFile and serveStatic for ESP8266/ESP32 boards

[AC8L]

Per your request refactored the code to server only ESP8266/ESP32 platforms. I spent whole day to create a class which works but request handlers are broken. So, ended up separating the code.

[khoih-prog]

Thanks for your good efforts. I'll have a more detailed look at the new PR within several days.

BTW, could you also provide the example code the test the new ESP streamFile feature to save me some time to write it to verify? I'll include it in the examples, with your credit, if the PR OK to be merged later.

[AC8L]

Done with the example sketch.

[khoih-prog]

Wow, you've spent a lot of time on the code.

I just had a preliminary look and test compiled the simplest example AdvancedWebServer and still there are many compiler errors

1. For boards different from ESP8288/ESP32, this is simple to fix by not using mimetable.cpp (using #if)

1. File mimetable.h

#ifndef __MIMETABLE_H__
#define __MIMETABLE_H__

#if (ESP32 || ESP8266)
...

#endif
#endif

2. File mimetable.cpp

#if (ESP32 || ESP8266)

...

#endif

The compiler errors

• 1.1 For SAM DUE

/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/detail/esp_detail/mimetable.cpp:2:22: fatal error: pgmspace.h: No such file or directory
 #include "pgmspace.h"
                      ^
compilation terminated.
Error compiling for board Arduino Due (Programming Port).

• 1.2 AVR Mega

/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/detail/esp_detail/mimetable.cpp:2:10: fatal error: pgmspace.h: No such file or directory
 #include "pgmspace.h"
          ^~~~~~~~~~~~
compilation terminated.
exit status 1
Error compiling for board Arduino Mega or Mega 2560.

• 1.3 SAMD51

/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/detail/esp_detail/mimetable.cpp:2:10: fatal error: pgmspace.h: No such file or directory
    2 | #include "pgmspace.h"
compilation terminated.
exit status 1
Error compiling for board Adafruit ItsyBitsy M4 (SAMD51).

• Teensy 4.0

/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/detail/esp_detail/mimetable.cpp:2:22: fatal error: pgmspace.h: No such file or directory
compilation terminated.
exit status 1
Error compiling for board Teensy 4.0.

---

2.Even compiler errors for ESP8288/ESP32, this is also simple to fix by using only ESP libb64, and not using libb64 of this library. Could you check and fix

I'm using ESP8266 core 2.7.4 and ESP32 core 1.0.4 with Arduino IDE v1.8.13

• 1.1 ESP8266

/home/kh/Downloads/Arduino/arduino-1.8.13/hardware/esp8266com/esp8266/cores/esp8266/libb64/cencode.cpp:13: multiple definition of `base64_init_encodestate'; libraries/EthernetWebServer-serveStatic_dev2/libb64/cencode.c.o:/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/libb64/cencode.c:44: first defined here
/home/kh/.arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/2.5.0-4-b40a506/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld: core/core.a(cencode.cpp.o): in function `base64_encode_value':
/home/kh/Downloads/Arduino/arduino-1.8.13/hardware/esp8266com/esp8266/cores/esp8266/libb64/cencode.cpp:25: multiple definition of `base64_encode_value'; libraries/EthernetWebServer-serveStatic_dev2/libb64/cencode.c.o:/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/libb64/cencode.c:50: first defined here
/home/kh/.arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/2.5.0-4-b40a506/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld: core/core.a(cencode.cpp.o): in function `base64_encode_block':
/home/kh/Downloads/Arduino/arduino-1.8.13/hardware/esp8266com/esp8266/cores/esp8266/libb64/cencode.cpp:41: multiple definition of `base64_encode_block'; libraries/EthernetWebServer-serveStatic_dev2/libb64/cencode.c.o:/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/libb64/cencode.c:60: first defined here
/home/kh/.arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/2.5.0-4-b40a506/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld: core/core.a(cencode.cpp.o): in function `base64_encode_blockend':
/home/kh/Downloads/Arduino/arduino-1.8.13/hardware/esp8266com/esp8266/cores/esp8266/libb64/cencode.cpp:97: multiple definition of `base64_encode_blockend'; libraries/EthernetWebServer-serveStatic_dev2/libb64/cencode.c.o:/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/libb64/cencode.c:137: first defined here
/home/kh/.arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/2.5.0-4-b40a506/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld: core/core.a(cencode.cpp.o): in function `base64_encode_chars':
/home/kh/Downloads/Arduino/arduino-1.8.13/hardware/esp8266com/esp8266/cores/esp8266/libb64/cencode.cpp:118: multiple definition of `base64_encode_chars'; libraries/EthernetWebServer-serveStatic_dev2/libb64/cencode.c.o:/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/libb64/cencode.c:161: first defined here
collect2: error: ld returned 1 exit status
exit status 1
Error compiling for board NodeMCU 1.0 (ESP-12E Module).

• ESP32

In file included from /home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/detail/RequestHandlersImpl.h:192:0,
/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/detail/ESP_RequestHandlersImpl.h:28:17: fatal error: Uri.h: No such file or directory
compilation terminated.
exit status 1
Error compiling for board ESP32 Dev Module.

[khoih-prog]

It's OK now for ESP8266, just wrap

#if !(ESP32 || ESP8266)

...
#endif

around cencode.c and cdecode.c solves the ESP compile issue.

Can you fix and update the PR?

[khoih-prog]

Still having issue with ESP32.

In file included from /home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/detail/RequestHandlersImpl.h:192:0,
/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/detail/ESP_RequestHandlersImpl.h:28:17: fatal error: Uri.h: No such file or directory
compilation terminated.
exit status 1
Error compiling for board ESP32 Dev Module.

[khoih-prog]

Fix now by comment out #include "Uri.h" in ESP_RequestHandlersImpl.h

#include "RequestHandler.h"
#include "esp_detail/mimetable.h"
#include "FS.h"
#include "WString.h"
//#include "Uri.h"
#include <MD5Builder.h>
#include <base64.h>

Pls update the PR.

[khoih-prog]

The example serveStatic, I think, is written for ESP32 and deprecated SPIFFS. I'll rewrite to include LittleFS for both ESP32 and ESP8266, also remove all unnecessary #defines.

[AC8L]

Made all suggested changes except for cencode.h which was not necessary. Tested by compiling AdvancedWebServer for SAMD and ESP boards without errors. Feel free editing the example file. After all it will be a good example for LittleFS as well.

[khoih-prog]

I had to modify cencode.c and cdecode.c to solve the ESP8266 compile issue. I think you compile only for ESP32, not ESP8266.

Can you check to be sure using the latest ESP cores and Arduino IDE ?

[AC8L]

On ESP8266 I am getting different error:

AdvancedWebServer:183:14: error: 'class EthernetClass' has no member named 'init'
Ethernet.init (USE_THIS_SS_PIN);
^
Multiple libraries were found for "Ethernet.h"
Used: /Users/sardar/Library/Arduino15/packages/esp8266/hardware/esp8266/2.7.4/libraries/Ethernet
Not used: /Applications/Arduino.app/Contents/Java/libraries/Ethernet
exit status 1

I am using Arduino IDE 1.8.14, ESP8266 core 2.7.4.
But for ESP32 I am using 1.0.5-rc6, since it fixes the python issue on. Mac OS Big Sur I am using.

[AC8L]

I tested the previous error - I am getting it even with the original code on ESP8266.

[khoih-prog]

Everything is OK here now. You have to check your settings / installation

I'm writing an example to permit you to upload the files (e.g. page1, etc.) into LittleFS/SPIFFS so that you can use with the staticServe example. Already working, will ready for next release, with your example and PR.

Also more complicated FS_EthernetWebServer example with Web editor.

Still have to test everything to see where else it breaks.

[khoih-prog]

The ESP8266 compile result (IDE v1.8.13, ESP8266 core 2.7.4)

In file included from /home/kh/Arduino/khoih-prog_working/Testing_Libs/EthernetWebServer-serveStatic_dev2/examples/AdvancedWebServer/AdvancedWebServer.ino:44:0:
sketch/defines.h:228:4: warning: #warning Use ESP8266 architecture [-Wcpp]
   #warning Use ESP8266 architecture
    ^
In file included from sketch/defines.h:328:0,
                 from /home/kh/Arduino/khoih-prog_working/Testing_Libs/EthernetWebServer-serveStatic_dev2/examples/AdvancedWebServer/AdvancedWebServer.ino:44:
/home/kh/Arduino/libraries/Ethernet/src/Ethernet.h:200:0: warning: "UDP_TX_PACKET_MAX_SIZE" redefined [enabled by default]
 #define UDP_TX_PACKET_MAX_SIZE 24
 ^
In file included from /home/kh/Downloads/Arduino/arduino-1.8.13/hardware/esp8266com/esp8266/libraries/ESP8266mDNS/src/ESP8266mDNS_Legacy.h:47:0,
                 from /home/kh/Downloads/Arduino/arduino-1.8.13/hardware/esp8266com/esp8266/libraries/ESP8266mDNS/src/ESP8266mDNS.h:45,
                 from sketch/defines.h:229,
                 from /home/kh/Arduino/khoih-prog_working/Testing_Libs/EthernetWebServer-serveStatic_dev2/examples/AdvancedWebServer/AdvancedWebServer.ino:44:
/home/kh/Downloads/Arduino/arduino-1.8.13/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/WiFiUdp.h:28:0: note: this is the location of the previous definition
 #define UDP_TX_PACKET_MAX_SIZE 8192
 ^
In file included from /home/kh/Arduino/khoih-prog_working/Testing_Libs/EthernetWebServer-serveStatic_dev2/examples/AdvancedWebServer/AdvancedWebServer.ino:44:0:
sketch/defines.h:329:8: warning: #warning Using Ethernet lib [-Wcpp]
       #warning Using Ethernet lib
        ^
In file included from sketch/defines.h:344:0,
                 from /home/kh/Arduino/khoih-prog_working/Testing_Libs/EthernetWebServer-serveStatic_dev2/examples/AdvancedWebServer/AdvancedWebServer.ino:44:
/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/EthernetWebServer.h:94:4: warning: #warning Using Ethernet library from EthernetWebServer [-Wcpp]
   #warning Using Ethernet library from EthernetWebServer
    ^
In file included from sketch/defines.h:344:0,
                 from /home/kh/Arduino/khoih-prog_working/Testing_Libs/EthernetWebServer-serveStatic_dev2/examples/AdvancedWebServer/AdvancedWebServer.ino:44:
/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/EthernetWebServer.h:109:4: warning: #warning SENDCONTENT_P_BUFFER_SZ using default 4 Kbytes [-Wcpp]
   #warning SENDCONTENT_P_BUFFER_SZ using default 4 Kbytes
    ^
Executable segment sizes:
IROM   : 266064          - code in flash         (default or ICACHE_FLASH_ATTR) 
IRAM   : 26904   / 32768 - code in IRAM          (ICACHE_RAM_ATTR, ISRs...) 
DATA   : 1664  )         - initialized variables (global, static) in RAM/HEAP 
RODATA : 2856  ) / 81920 - constants             (global, static) in RAM/HEAP 
BSS    : 25216 )         - zeroed variables      (global, static) in RAM/HEAP 
Sketch uses 297488 bytes (28%) of program storage space. Maximum is 1044464 bytes.
Global variables use 29736 bytes (36%) of dynamic memory, leaving 52184 bytes for local variables. Maximum is 81920 bytes.

[khoih-prog]

Ah ! Remember now. You have to rename the file Ethernet.h to Ethernet_8266.h in ./hardware/esp8266com/esp8266/libraries/Ethernet directory.

I think I forget to mention in the README about this to avoid conflict.

[AC8L]

You mentioned it in defines.h for example sketches but it is in the middle of define statements and looks like something optional :) I think mentioning it in readme will be better.

[khoih-prog]

OK, here it is. Will include in README later.

// Note: To rename ESP628266 Ethernet lib files to Ethernet_ESP8266.h and Ethernet_ESP8266.cpp
// In order to USE_ETHERNET_ESP8266

[AC8L]

Updated encode.c and cdecode.c files.

And thank you for the great example, it looks gorgeous!

[khoih-prog]

I think everything is ready to merge the PR now, but I'd still like to have more compiling and testing to be sure no more issue before making final decision to merge.

Can you help compile and test as many as you can, on whatever board you can? With your quick decision and good knowledge, I wish later you can help maintain this library.

I'll post the new example code for you to test as well. Almost done.

[AC8L]

No rush, let's test throroughly.

[khoih-prog]

I just post the first 2 examples

1. serveStatic
2. serveStaticLoadFile

Can you test, then update your repo to avoid conflict in PR.

[khoih-prog]

Congrats. The new EthernetWebServer v1.3.0 has just been published, thanks to your PR and contribution.

---

Major Releases v1.3.0

1. Add WebServer feature to serve from LittleFS/SPIFFS for ESP32/ESP8266 with examples. Check streamFile and serveStatic for ESP8266/ESP32 boards #22
2. Add examples serveStatic and serveStaticLoadFile to use new serveStatic()feature
3. Add examples ESP32_FS_EthernetWebServer and FS_EthernetWebServer to use new serveStatic()feature

---

Your contribution has been noted in Contributions and Thanks.

---

Please have more tests, report issues and make more PRs.

Best Regards,