-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add neurosift-dependent streaming libraries #30
Comments
If this is running JavaScript, do you think that maybe PyExecJS or the js2py library would work to run the script via Python? |
@CodyCBakerPhD This is not so straightforward since neurosift uses WebAssembly in a WebWorker, and as far as I know you can only run web workers within a browser. But there could be a way to refactor this library so that the code can be run in the main thread of nodejs. Since it uses webassembly, js2py is not going to work. I think the best bet would be to run using nodejs as a system command. Will require some refactoring. |
Would it be possible to write a really simple standalone that incorporates in-line timing around the lines of interest and dumps the information to a results file (JSON) of the structure we expect from the rest of the suite? If so then we can have Python based CLI triggers to 'open' that HTML (not sure about closing, though. Can a page automatically close itself after a certain delay?) |
I don't think this is feasible. For one thing, there's no way to get the web page to write to a .json file. The better option would be to write a nodejs program, because that executes in a normal process and has access to the file system. |
If we wanted to ensure that we were benchmarking in the browser (to avoid differences between that and Node.js), we could create a WebSocket system that uses a Python server to request actions on the HTML client and which receives the execution information back. We could also use my commoners library to create an Electron application (run for development purposes only) that could be executed with Though if we don't care about the fact that |
Now that I'm thinking about it, you could achieve this without Electron or WS by using |
Is there a technological difference that could produce different results between running the same |
I think the results should be comparable. However, I just took a crack at creating a node version of this, and it seems it's going to be difficult to get that to work. So maybe one of @garrettmflynn 's two suggestions would be the way to go. |
Here's a basic sandbox for benchmarking on the web and getting timing results back in Python: https://github.com/garrettmflynn/webnwb-benchmark Looks like we won't be able to time using Python directly here because of some variability inherent in using Puppeteer |
Together with #27 and #28 we should try to see if we can add benchmark capability of the
h5wasm
andremote-h5-file
on the same files and slice selections as the other methods (for rigorous comparison between the two)@magland @garrettmflynn Do either of you know if it's possible to have a javascript service running on a background process that can be triggered by a Python call in a way that can track the amount of time it took for a given operation to occur
For example, I want to perform the analog of the following operation using
remfile
: https://github.com/NeurodataWithoutBorders/nwb_benchmarks/blob/refactor_functional_fully_explicit/src/nwb_benchmarks/benchmarks/time_remote_slicing.py#L45-L58 (where thesetup
is not a part of the timing/tracking but the actual data slice is)but via the two neurosift-dependent libraries mentioned above
The text was updated successfully, but these errors were encountered: