Web platform support using emscripten

[GPMueller]

First of all, thanks for this extremely useful project!
I would like to have it run in the browser by compiling it to web-assembly using emscripten. Emscripten itself does not currently provide support for std::filesystem and it seems to me this might be in scope for your library.

I was able to have it compile using emscripten by simply adding

#elif defined(__EMSCRIPTEN__)
#define GHC_OS_WEB

to your platform determination part in filesystem.hpp.

It was immediately usable, but the following simple example gives a crash, so I suppose there may be some more effort necessary to get things running correctly:

for(auto& p: fs::recursive_directory_iterator("."))
        std::cout << p.path() << '\n';

fails after two prints

./proc
./proc/self
Operation not permitted: './proc/self'

Note that the "filesystem" in this case is the virtual browser filesystem -- I have not yet researched the nuances that might make the browser platform different from others.

[gulrak]

Never tried to build and use it using emscripten, but a permission error during recursive iteration is perfectly possible on any of the supported platforms, depending on the rights. Did you try:

for(auto& p: fs::recursive_directory_iterator(".", fs::directory_options::skip_permission_denied))
        std::cout << p.path() << '\n';

[GPMueller]

Strange, but it gives the same result on my test setup.

I haven't figured out yet how to run emscripten-compiled unit tests inside a headless browser and get the output back (i.e. to run them in CI). If that were to work, one could use your tests in CI to determine where any web-specific changes might be needed.

[gulrak]

I gave it a quick try to build the tests with emscripten and there are quite some issues, as not all features are supported (hard links are one example) so some of the tests need to be disabled for support, and some need investigation. Explicitly running the recursive_directory_iterator test gives it a pass in all 39 assertions, so nothing obviously wrong. The most important thing I needed to do to run the tests was compiling/linking with -s DISABLE_EXCEPTION_CATCHING=0 (I didn't try the other option -s DISABLE_EXCEPTION_CATCHING=2) as ghc::filesystem uses exceptions like the std::filesystem by default.

I added that code fragment to the test and get the same Operation not permitted: './proc/self' response, no matter if I add skip_permissions_denied or not. So there is something unexpected happening. Not sure if I find enough time this evening to investigate further, but I try.

[gulrak]

I found some situation where the condition of permission error comes later in the loop, and that might be something that can hit other platforms too.

Sadly the error code returned by the emscripten environment is 63 with "Operation not permitted" as the message. The error Code for EPERM is 1, I would also expect EACCES, so it will not be detected correctly even with that potential fix. I have yet to find a an __EMSCRIPTEN__ dependent include that matches this error with a detectable error because I sure will not add numeric 63 to that, when it is normally reserved for ENOSR meaning out of streams resources.

Looks like I need input from someone more into emscripten.

[gulrak]

Okay, actually that last comment was my fault. So I was able to hot-fix this locally and get

"./tmp"
"./home"
"./home/web_user"
"./dev"
"./dev/null"
"./dev/tty"
"./dev/tty1"
"./dev/random"
"./dev/urandom"
"./dev/shm"
"./dev/shm/tmp"
"./dev/stdin"
"./dev/stdout"
"./dev/stderr"
"./proc"
"./proc/self"

on my system, without an error.

I'm currently on my way into office, so I can't work on the real fix until this evening. The commit will only fix the iteration issue ignoring the skip_permissions_denied flag. A real emscripten support is more than I can manage without more time.

[gulrak]

Work on experimental support for use with emscripten is taking place on branch feature-68-experimental-emscripten-support. Most tests are working now, hard link related features are disabled, there are still issues with fs::last_write_time and some smaller hiccups.

[gulrak]

Changes released with v1.3.4

[GPMueller]

Awesome 👍 thanks!