WDL to extract Avro files for Hail import [VS-579] #7981
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## ah_var_store #7981 +/- ##
================================================
Coverage ? 86.247%
Complexity ? 35205
================================================
Files ? 2173
Lines ? 165016
Branches ? 17792
================================================
Hits ? 142321
Misses ? 16368
Partials ? 6327 |
gbggrant
approved these changes
Aug 9, 2022
|
Successful run linked in the description. |
|
Thanks - missed that. |
|
I missed exporting the tranche data 🙈, fixes incoming |
RoriCremer
approved these changes
Aug 10, 2022
RoriCremer
reviewed
Aug 10, 2022
| bq query --nouse_legacy_sql --project_id=~{project_id} " | ||
| EXPORT DATA OPTIONS( | ||
| uri='${avro_prefix}/vqsr_tranche/vqsr_tranche_*.avro', format='AVRO', compression='SNAPPY') AS | ||
| SELECT model, truth_sensitivity, min_vqslod, filter_name |
There was a problem hiding this comment.
I think we're going to want to drop filter_name here, but I'm adding it to the questions for Tim this afternoon
RoriCremer
reviewed
Aug 10, 2022
| } | ||
|
|
||
| # Superpartitions have max size 4000. The inner '- 1' is so the 4000th (and multiples of 4000) sample lands in the | ||
| # appropriate partition, the outer '+ 1' is to iterate over the correct number of partitions. |
There was a problem hiding this comment.
This is the only thing that I'd want to see tested out (before the tie out) and I'm not sure of the best way to do this...maybe this can wait until we do this with 10k samples
This was referenced Mar 17, 2023
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Successful run here.