X/Y chromosome reweighting for better extract shard runtime balance [VS-389] #7868
Conversation
mcovarr
changed the title
mcovarr
changed the title
Codecov Report
@@ Coverage Diff @@
## ah_var_store #7868 +/- ##
================================================
Coverage ? 86.296%
Complexity ? 35191
================================================
Files ? 2170
Lines ? 164876
Branches ? 17784
================================================
Hits ? 142281
Misses ? 16271
Partials ? 6324 |
mcovarr
force-pushed
the
vs_389_xy_reweighting
branch
from
0b538ff
to
5894be5
Compare
| Float y_bed_weight_scaling | ||
| } | ||
| command <<< | ||
| python3 <<FIN |
There was a problem hiding this comment.
I should possibly make this a separate script with tests and add it to the Docker image.
| @@ -404,16 +404,6 @@ private static long getQueryCostBytesProcessedEstimate(String queryString, Strin | |||
| return bytesProcessed; | |||
| } | |||
|
|
|||
| public static StorageAPIAvroReader executeQueryWithStorageAPI(final String queryString, | |||
There was a problem hiding this comment.
drive by removal of unneeded method
|
|
||
| if line.startswith('chrX') or line.startswith('chrY'): | ||
| scale_factor = x_scale_factor if line.startswith('chrX') else y_scale_factor | ||
| fields = line.split('\t') |
There was a problem hiding this comment.
Might want to just validate that the bed has five fields here? - and throw an exception if not.
(not sure if there's any optional columns that could be in a bed, in which case your fields[-1] might be updating the wrong field).
| @@ -404,16 +404,6 @@ private static long getQueryCostBytesProcessedEstimate(String queryString, Strin | |||
| return bytesProcessed; | |||
| } | |||
|
|
|||
| public static StorageAPIAvroReader executeQueryWithStorageAPI(final String queryString, | |||
| - name: GvsIngestTieout | ||
| subclass: WDL | ||
| primaryDescriptorPath: /scripts/variantstore/wdl/GvsIngestTieout.wdl | ||
| filters: | ||
| branches: | ||
| - master | ||
| - ah_var_store | ||
| - vs_261_ingest_errors |
There was a problem hiding this comment.
if I had looked at this pr earlier, I would have seen that you already did this! Thank you!!!!!
| @@ -21,13 +21,11 @@ workflow GvsCreateFilterSet { | |||
| String? service_account_json_path | |||
| Int? SNP_VQSR_max_gaussians_override = 6 | |||
| Int? SNP_VQSR_mem_gb_override | |||
| # this is the minimum number of samples where the SNP model will be created and applied in separate tasks | |||
| # (SNPsVariantRecalibratorClassic vs. SNPsVariantRecalibratorCreateModel and SNPsVariantRecalibratorScattered) | |||
| Int snps_variant_recalibration_threshold = 5000 | |||
There was a problem hiding this comment.
I agree with the change, but it might be worth noting that for WARP classic this is done with 20k. Probably not necessary since it's purely implementation, but I know we've been trying to stay as true to WARP as possible
mcovarr
force-pushed
the
vs_389_xy_reweighting
branch
from
a4299b2
to
eb2b973
Compare
No description provided.