Deprecate SPIFFS, move examples to LittleFS

[earlephilhower]

SPIFFS has been a great filesystem, but it has significant problems in
many cases (and it's also pretty slow). Development seems to have
slowed/stopped on the upstream version, and we're not able to provide
support or fix the known issues with it as-is.

Deprecate SPIFFS variable.

Update all examples to use LittleFS instead of SPIFFS.

Also, minor cleanup on very old examples which has obsolete delays
waiting for the Serial port to come up, or which were stuck at 9600 baud
because of their ancient AVR heritage.

Fixes #7095

[earlephilhower]

Apps which use SPIFFS will get warnings like the following, but will still run fine:

In file included from C:\Users\earle\Documents\Arduino\hardware\esp8266com\esp8266\cores\esp8266\spiffs_api.cpp:24:0:
C:\Users\earle\Documents\Arduino\hardware\esp8266com\esp8266\cores\esp8266\spiffs_api.h: In member function 'virtual bool spiffs_impl::SPIFFSImpl::begin()':
C:\Users\earle\Documents\Arduino\hardware\esp8266com\esp8266\cores\esp8266\spiffs_api.h:188:21: warning: 'void spiffs_impl::SPIFFSImpl::_deprecated()' is deprecated (declared at C:\Users\earle\Documents\Arduino\hardware\esp8266com\esp8266\cores\esp8266\spiffs_api.h:93): SPIFFS use is deprecated. Please consider moving to LittleFS. [-Wdeprecated-declarations]
         _deprecated();
                     ^

[devyte]

Fixes #7095

[earlephilhower]

The new deprecation warning, shows the direct line SPIFFS is used:

C:\Users\earle\AppData\Local\Temp\arduino_modified_sketch_35942\LittleFS_Timestamp.ino: In function 'bool getLocalTime(tm*, uint32_t)':
C:\Users\earle\AppData\Local\Temp\arduino_modified_sketch_35942\LittleFS_Timestamp.ino:23:3: warning: 'SPIFFS' is deprecated (declared at C:\Users\earle\Documents\Arduino\hardware\esp8266com\esp8266\cores\esp8266/FS.h:269): SPIFFS has been deprecated. Please consider moving to LittleFS or other filesystems. [-Wdeprecated-declarations]
   SPIFFS.begin();
   ^

[devyte]

Too bad about the speedtest, I've actually used a variation of that to check a module's FS. Could that sketch maybe be refactored for "some" FS defined at the top? I.e.: change a define to use for LittleFS or SD or XYZ, with LittleFS being the default?
Approving anyways.

[earlephilhower]

@devyte, I can leave in the SpeedTest.ino. But, since there isn't another supported internal FS, I'm really not sure what the purpose would be. Let me know and I can re-add.

[devyte]

The original intent was to contrast SPIFFS vs LittleFS performance in one run. Even without SPIFFS, this could serve to contrast e. g. LittleFS vs SD, or SD with one type of card vs another type of card, by running the sketch multiple times and collecting the results.
I suggest leaving the sketch, but simplify to one specific FS (leave a single DoTest call), and maybe add some mechanism at the top to choose which FS to use, like a macro or something, to make it easy to use. E. g. comment one line and uncomment another and it's SD instead of LittleFS.

[denis-stepanov]

This should have been done in a minor or major release, not in maintenance.

[earlephilhower]

It was intended for 2.7.0 but didn't make it to the cutoff. We're not killing it in 2.7.1, just giving everyone heads-up. You can still use your SPIFFS setup if desired in 2.7.1.

[intensite]

This should have been done in a minor or major release, not in maintenance.

I am also concerned about this change. Not against it however.
Is LittleFS a "drop-in/transparent" replacement? Or do we need to rewrite everything that uses SPIFFS?

How about the "Sketch Data Upload" function of the Arduino IDE, is there a similar tool to copy files from PC to ESP32's persistant memory?

[d-a-v]

LittleFS is indeed a drop-in / transparent replacement for SPIFFS. Just replace all occurrence of SPIFFS by LittleFS in your code.
However, FS will be reformated and there is no procedure to transform an already existing SPIFFS FS into a LittleFS one.

About uploading: https://arduino-esp8266.readthedocs.io/en/latest/filesystem.html?highlight=LittleFS#uploading-files-to-file-system

[joysfera]

This filesystem deprecation is kinda crazy considering that there are millions of devices with important data on SPIFFS out there. Without a semi-automated and safe way of transforming SPIFFS to LittleFS (if it's possible at all, probably not in a truly safe way) people will have to either ignore the deprecation warning or ignore new releases.

[earlephilhower]

Fair points, @joysfera, but deprecation doesn't mean we're dropping it immediately.

No new designs should be using SPIFFS due to a bunch of known problems and low performance. It's not really supported upstream, and we're not equipped to do the maintenance ourselves. So if it works, great. If there's a problem, we're not going to spend time on it (but will always accept PRs!)...

[devyte]

@joysfera be that as it may, SPIFFS has known issues and limitations, and given the lack of activity of the upstream repo in the past 3 years, they're unlikely to be addressed there any time soon. We won't investigate SPIFFS related issues due to upstream code, and there's no upstream maintainer, so deprecation it is. As for migration, it's as simple as downloading the files from SPIFFS to some local storage, updating the sketch to LittleFS, and then uploading either the files or a LittleFS image made with the files. Exactly how that migration is implemented depends on each individual user app or 3rd party infrastructure, so it can't be specific to the core.
Users are free to ignore the deprecation for now, there is ample time to migrate, but as I said we won't accept SPIFFS related issues moving forward (unless they're accompanied by a PR). If at some point LittleFS overcomes its current limitations, and SPIFFS upstream doesn't reactivate, we will eventually pull SPIFFS, though.
Users are also free to ignore new releases, but then they have to live with the consequences of lack of support as the repo moves forward with newer versions.

[joysfera]

You seem to forget about the devices in the field, not on developers' desks. Those are not accessible, you cannot download their files to some local storage, reformat the FS and upload the files back. Think millions of Sonoff devices, for example.
I understand your points, I don't ask you for supporting the SPIFFS, I see that you want to advise new users to skip SPIFFS and start with LittleFS instead. But removing the SPIFFS would be a backward incompatible step, and quite damaging for all the deployed devices, I am afraid.
If there was an OTA upgrade path, at least...

[joysfera]

Example of an automatic upgrade path:

1. find out the size of the filesystem and the total length of all files
2. if the total length of all files is lower than half of the size of the filesystem we have a chance
3. shrink the existing SPIFFS (requires defragmentation first :-O) to half of its original size
4. create LittleFS in the freed space and format it
5. copy all files from SPIFFS to LittleFS
6. make sure that the copies are OK (compare their SHA checksums)
7. switch the OTA updated code to start using LittleFS

Questions: is it possible to shrink SPIFFS? Is it possible to locate LittleFS in the freed space?

Issues:

1. defragmentation cannot be done safely
2. the SPIFFS filesystem might be too full so it does not allow shrinking
3. there might not be enough code space for two filesystems and a code for defragmentation and copying of all files.

[earlephilhower]

Yes, @joysfera , that sounds strikingly like the process I remember Windows 95(?) going through when converting FAT32 to NTFS. We'd be happy to look at any PR you come up with!

You might want to chime in on littlefs-project/littlefs#238 where someone is going through a similar exercise.