pod5 view does not work for some data since version 0.3.0 #107
Comments
|
Sorry, only now I saw issue #106, opened yesterday by roo-weed. I am facing the same problem that him. I am not sure if it is possible to join the issues, but the solution to one will certainly be the solution to the other. My apologies. Best regards. -- |
|
Hi @davidsilvapires, thanks for the detailed report. Can you report which version of Kind regards, |
|
Hello @HalfPhoton. Yes, of course. Each time I reinstall pod5, a different version of Bellow you can find all the messages that I receive after each reinstallation of pod5: Thank you very much for helping me with this issue. -- |
|
Could you please try pod5 0.3.6 with Kind regards, |
|
Sure. It worked, @HalfPhoton. Thank you very much! I didn't set the polars version at pod5 reinstallation. The installed version was polars~=0.19. And then, the pod5 view command worked. See below the complete output: Have you already updated the code at GitHub? I ask because I tried the same commands yesterday from a fresh install and pod5 view wasn't working. I created a new virtual environment now and installed pod5 with just And the final message: So, I confirm that everything works with the most updated version available right now. I am very grateful for you support. Thank you very much, @HalfPhoton. Kind regards. -- |
|
Hi @davidsilvapires , I thought we'd fixed this issue in I'll leave this ticket open until we push a patch. Kind regards, |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi, everybody!
It seems that the subcommand "view" of the command "pod5" does not work for all kind of Nanopore data. See, for example, the following case:
curl -OL 'https://sra-pub-src-1.s3.amazonaws.com/SRR23640421/barcode01.tar.gz.gz.1'
tar zxvvf barcode01.tar.gz.gz.1
pip install pod5==0.2.4
pod5 convert fast5 barcode01/*.fast5 --threads 8 --output barcode01.pod5
pod5 view barcode01.pod5 --threads 8 --include 'read_id, channel' --output summary.tsv
With version 0.2.4, the command works fine. But from version 0.3.0 and above:
pip install pod5==0.3.0
rm barcode01.pod5 summary.tsv
pod5 convert fast5 barcode01/*.fast5 --threads 8 --output barcode01.pod5
pod5 view barcode01.pod5 --threads 8 --include 'read_id, channel' --output summary.tsv
the following error is returned:
POD5 has encountered an error: 'Error while processing 'barcode01.pod5''
For detailed information set POD5_DEBUG=1'
If we set the environment variable:
export POD5_DEBUG=1
rm summary.tsv
pod5 view barcode01.pod5 --threads 8 --include 'read_id, channel' --output summary.tsv
polars.exceptions.ColumnNotFoundError: not_set
Error originated just after this operation:
WITH_COLUMNS:
[String(barcode01.pod5).alias("filename"), col("read_id").bin.hex_encode().str.slice().str.concat_horizontal([String(-), col("read_id").bin.hex_encode().str.slice(), String(-), col("read_id").bin.hex_encode().str.slice(), String(-), col("read_id").bin.hex_encode().str.slice(), String(-), col("read_id").bin.hex_enc
ode().str.slice()]), col("well").alias("mux"), col("num_minknow_events").alias("minknow_events"), col("experiment_name").alias("experiment_id"), [(col("start")) / (col("sample_rate"))].alias("start_time"), [(col("num_samples")) / (col("sample_rate"))].alias("duration")]
INNER JOIN:
LEFT PLAN ON: [col("run_info")]
UNION
PLAN 0:
WITH_COLUMNS:
[col("run_info").strict_cast(String)]
DF ["read_id", "read_number", "start", "median_before"]; PROJECT */20 COLUMNS; SELECTION: "None"
PLAN 1:
WITH_COLUMNS:
[col("run_info").strict_cast(String)]
DF ["read_id", "read_number", "start", "median_before"]; PROJECT */20 COLUMNS; SELECTION: "None"
PLAN 2:
(...)
RIGHT PLAN ON: [col("acquisition_id")]
UNIQUE BY None
UNIQUE BY None
DF ["acquisition_id", "acquisition_start_time", "adc_max", "adc_min"]; PROJECT */18 COLUMNS; SELECTION: "None"
END INNER JOIN
Error originated just after this operation:
ErrorStateSync(AlreadyEncountered(not found: not_set
Error originated just after this operation:
WITH_COLUMNS:
[String(barcode01.pod5).alias("filename"), col("read_id").bin.hex_encode().str.slice().str.concat_horizontal([String(-), col("read_id").bin.hex_encode().str.slice(), String(-), col("read_id").bin.hex_encode().str.slice(), String(-), col("read_id").bin.hex_encode().str.slice(), String(-), col("read_id").bin.hex_enc
ode().str.slice()]), col("well").alias("mux"), col("num_minknow_events").alias("minknow_events"), col("experiment_name").alias("experiment_id"), [(col("start")) / (col("sample_rate"))].alias("start_time"), [(col("num_samples")) / (col("sample_rate"))].alias("duration")]
INNER JOIN:
LEFT PLAN ON: [col("run_info")]
UNION
PLAN 0:
WITH_COLUMNS:
[col("run_info").strict_cast(String)]
DF ["read_id", "read_number", "start", "median_before"]; PROJECT */20 COLUMNS; SELECTION: "None"
PLAN 1:
WITH_COLUMNS:
[col("run_info").strict_cast(String)]
DF ["read_id", "read_number", "start", "median_before"]; PROJECT */20 COLUMNS; SELECTION: "None"
PLAN 2:
(...)
RIGHT PLAN ON: [col("acquisition_id")]
UNIQUE BY None
UNIQUE BY None
DF ["acquisition_id", "acquisition_start_time", "adc_max", "adc_min"]; PROJECT */18 COLUMNS; SELECTION: "None"
END INNER JOIN))
WITH_COLUMNS:
[String(barcode01.pod5).alias("filename"), col("read_id").bin.hex_encode().str.slice().str.concat_horizontal([String(-), col("read_id").bin.hex_encode().str.slice(), String(-), col("read_id").bin.hex_encode().str.slice(), String(-), col("read_id").bin.hex_encode().str.slice(), String(-), col("read_id").bin.hex_enc
ode().str.slice()]), col("well").alias("mux"), col("num_minknow_events").alias("minknow_events"), col("experiment_name").alias("experiment_id"), [(col("start")) / (col("sample_rate"))].alias("start_time"), [(col("num_samples")) / (col("sample_rate"))].alias("duration")]
INNER JOIN:
LEFT PLAN ON: [col("run_info")]
UNION
PLAN 0:
WITH_COLUMNS:
[col("run_info").strict_cast(String)]
DF ["read_id", "read_number", "start", "median_before"]; PROJECT */20 COLUMNS; SELECTION: "None"
PLAN 1:
WITH_COLUMNS:
[col("run_info").strict_cast(String)]
DF ["read_id", "read_number", "start", "median_before"]; PROJECT */20 COLUMNS; SELECTION: "None"
PLAN 2:
(...)
RIGHT PLAN ON: [col("acquisition_id")]
UNIQUE BY None
UNIQUE BY None
DF ["acquisition_id", "acquisition_start_time", "adc_max", "adc_min"]; PROJECT */18 COLUMNS; SELECTION: "None"
END INNER JOIN
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/storage/zuleika/volume3/project/jcunha/hiChromatin/local/src/venv/pod5/bin/pod5", line 8, in
sys.exit(main())
File "/storage/zuleika/volume3/project/jcunha/hiChromatin/local/src/venv/pod5/lib/python3.8/site-packages/pod5/tools/main.py", line 60, in main
return run_tool(parser)
File "/storage/zuleika/volume3/project/jcunha/hiChromatin/local/src/venv/pod5/lib/python3.8/site-packages/pod5/tools/parsers.py", line 41, in run_tool
raise exc
File "/storage/zuleika/volume3/project/jcunha/hiChromatin/local/src/venv/pod5/lib/python3.8/site-packages/pod5/tools/parsers.py", line 38, in run_tool
return tool_func(**kwargs)
File "/storage/zuleika/volume3/project/jcunha/hiChromatin/local/src/venv/pod5/lib/python3.8/site-packages/pod5/tools/parsers.py", line 744, in run
return view_pod5(**kwargs)
File "/storage/zuleika/volume3/project/jcunha/hiChromatin/local/src/venv/pod5/lib/python3.8/site-packages/pod5/tools/utils.py", line 59, in wrapper
raise exc
File "/storage/zuleika/volume3/project/jcunha/hiChromatin/local/src/venv/pod5/lib/python3.8/site-packages/pod5/tools/utils.py", line 56, in wrapper
ret = func(*args, **kwargs)
File "/storage/zuleika/volume3/project/jcunha/hiChromatin/local/src/venv/pod5/lib/python3.8/site-packages/pod5/tools/pod5_view.py", line 535, in view_pod5
launch_view_workers(
File "/storage/zuleika/volume3/project/jcunha/hiChromatin/local/src/venv/pod5/lib/python3.8/site-packages/pod5/tools/pod5_view.py", line 489, in launch_view_workers
join_workers(processes, exceptions_queue)
File "/storage/zuleika/volume3/project/jcunha/hiChromatin/local/src/venv/pod5/lib/python3.8/site-packages/pod5/tools/pod5_view.py", line 395, in join_workers
raise RuntimeError(f"Error while processing '{path}'") from exc
RuntimeError: Error while processing 'barcode01.pod5'
Can you figure out what is the problem?
Thanks in advance for any help.
Best regards.
--
David
The text was updated successfully, but these errors were encountered: