-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error running LINDI in a subproccess #61
Comments
In this case, does the s3_url point to an hdf5 file in the bucket? If so, that's not going to work... you need to point to the corresponding prepared json file. If it's a public dandi file, I can provide a formula for determining the proper url. |
The parameters are: ''' I.e., for now I am just using one of the examples from the README to check that the benchmarks are working. We are generating LINDI files for the actual test datasets we are planning to use. I used the version of lindi from PiPy. Let me know in case I should use a different file or version of lindi. It looks like the read itself is working, ie, we are getting back an array with the expected shape from the slicing. It looks like the error is occurring in the teardown afterwards when the subprocess is being cleaned up. |
I am not able to reproduce. See below. I am on Ubuntu, and I installed nwb_benchmarks in a conda environment as in the docs. What OS are you using?
tag @CodyCBakerPhD EDIT: I also tried this on github codespaces and did not see any error |
I am running on MacOS 12.5.1 (monterey) with a M1 Max chip |
Also replicated on Windows 11, but I'll see if I can't make a minimal example and get it in a CI |
Unfortunately I don't have access to mac or windows right now... |
@oruebel I pushed a new version of lindi to pypi. While I don't understand what could be causing the problem, there is always a chance that something may have been accidentally fixed. :) So it's worth a try to run your tests again with version 0.3.2. |
Thanks, I'll give it a try |
This appears to no longer be an issue |
@magland @rly we are working on the nwb-benchmarks and am encountering a strange issue NeurodataWithoutBorders/nwb_benchmarks#57 Do you have an idea? It looks like this might be related to running LINDI in a subprocess. The particular benchmark loads data from a remote file by slicing into a LINDI dataset. Simply opening a file with LINDI (incl. read with PyNWB) seems to work. The relevant code in the benchmark suite is: https://github.com/NeurodataWithoutBorders/nwb_benchmarks/blob/main/src/nwb_benchmarks/benchmarks/time_remote_slicing.py#L139-L156 which also uses these helper functions to open the file https://github.com/NeurodataWithoutBorders/nwb_benchmarks/blob/fb464f1a7ab6a2c4b4ce91b798ac68b088cf514d/src/nwb_benchmarks/core/_streaming.py#L205-L226
The text was updated successfully, but these errors were encountered: