[VS-693] Add support for VQSR Lite to GvsCreateFilterSet #8157

koncheto-broad · 2023-01-12T15:05:59Z

Added a new argument use_classic_VQSR to the WDL to determine whether we are using the old VQSR or the new VQSR-Lite. It defaults to true, so it should not change the default behavior of other code.

Ran the code with it set to true for the old behavior (https://app.terra.bio/#workspaces/gvs-dev/GVS%20Tiny%20Quickstart%20hatcher/job_history/adbbc5dc-6168-47a6-8f76-a232b8b4b9db)

and false for the new behavior (https://app.terra.bio/#workspaces/gvs-dev/GVS%20Tiny%20Quickstart%20hatcher/job_history/e5f46969-f80d-411a-90b0-309ea46753b3)

and it appears to behave as expected for both cases. Screenshot of the table created in BQ

This branch is based upon Bec's original PoC branch, so will be pulling in that code to ah_var_store as well

Edit: This is a large PR, so it may be helpful to think of it as 3 separate pieces.

There was Sam's code, which was verified and code reviewed and merged into master. Afterwards, it was cherrypicked into an offshoot of ah_var_store. I do not believe we have to review that. If you want to see those though, you can see them here (with the caveat that this is a comparison against the current ah_var_store, which may be slightly misleading): ah_var_store...c95945c
There's Bec's initial poc code. This was part of our previous spike, and what I built upon. A link to just those changes is here: c95945c...3c19b50
There's my code, which was specifically to add in the pathway to choose between VQSR and VQSR-Lite. A link to just those changes is here: 3c19b50...3320837

…annotations. (#7954) * Adds wdl that tests joint VCF filtering tools (#7932) * adding filtering wdl * renaming pipeline * addressing comments * added bash * renaming json * adding glob to extract for extra files * changing dollar signs * small comments * Added changes for specifying model backend and other tweaks to WDLs and environment. * Added classes for representing a collection of labeled variant annotations. * Added interfaces for modeling and scoring backends. * Added a new suite of tools for variant filtering based on site-level annotations. * Added integration tests. * Added test resources and expected results. * Miscellaneous changes. * Removed non-ASCII characters. * Added documentation for TrainVariantAnnotationsModel and addressed review comments. Co-authored-by: meganshand <[email protected]>

…eous minor fixes to new annotation-based filtering tools. (#8049)

…input in JointVcfFiltering WDL (#8027) * Small changes to JointVCFFiltering WDL * making default for use_allele_specific_annotations * addressing comments

… including Java changes necessary to produce the different tsv files

codecov · 2023-01-12T15:27:58Z

Codecov Report

❗ No coverage uploaded for pull request base (ah_var_store@9cf8021). Click here to learn what that means.
The diff coverage is n/a.

Additional details and impacted files

@@               Coverage Diff                @@
##             ah_var_store     #8157   +/-   ##
================================================
  Coverage                ?   86.184%           
  Complexity              ?     35511           
================================================
  Files                   ?      2191           
  Lines                   ?    166339           
  Branches                ?     17900           
================================================
  Hits                    ?    143358           
  Misses                  ?     16593           
  Partials                ?      6388

gatk-bot · 2023-01-12T16:00:18Z

Github actions tests reported job failures from actions build 3903475240
Failures in the following jobs:

Test Type	JDK	Job ID	Logs
conda	8	3903475240.3	logs

scripts/variantstore/wdl/GvsCreateFilterSet.wdl

cleaning up verbiage

gatk-bot · 2023-01-17T19:21:31Z

Github actions tests reported job failures from actions build 3942175794
Failures in the following jobs:

Test Type	JDK	Job ID	Logs
conda	8	3942175794.3	logs

mcovarr

There is a lot in this PR. Should we be reviewing these changes in their entirety or just the most recent changes our team has made to integrate VQSR-Lite? If the former (which might be appropriate if we are going to be co-owners of this code), we might want to get a walkthrough by the authors.

scripts/gatkcondaenv.yml.template

scripts/vcf_site_level_filtering_cromwell_tests/README.md

…ut by gCNV and updated sklearn. (#7261)

rsasch

LGTM. Only thing that gives me pause is keeping everything to create the filter set in one WDL. GvsCreateFilterSet.wdl already had one fork (are the number of samples over a certain threshold?) and this adds another so there's a lot of code that isn't going to get run in any given submission. I would prefer (not a dealbreaker) two additional WDLs that can be run as sub-workflows of GvsCreateFilterSet.wdl to generate the filter set depending on what use_classic_VQSR is set to (this will also make it easier if we decide to ditch one).

scripts/vcf_site_level_filtering_cromwell_tests/run_vcf_site_level_filtering_wdl.sh

scripts/vcf_site_level_filtering_wdl/JointVcfFiltering.wdl

scripts/variantstore/wdl/GvsCreateFilterSet.wdl

mcovarr · 2023-01-25T23:43:05Z

scripts/variantstore/wdl/GvsCreateFilterSet.wdl

+        sites_only_vcf = MergeVCFs.output_vcf,
+        sites_only_vcf_index = MergeVCFs.output_vcf_index,
+        basename = filter_set_name,
+        gatk_docker = "us.gcr.io/broad-gatk/gatk:4.3.0.0",


Curious why the GATK image is hardcoded here? Alternatively it could be an input with this as the default value.

Good question. It looks as though that was necessary when Bec yanked the code from master (which is at GATK 4.3.0) into our branch. It may have been dependent on the latest version of GATK? Maybe we leave it as an input with that as the default value and see if we can remove it if/when our branch catches up to master.

mcovarr · 2023-01-25T23:49:10Z

scripts/variantstore/wdl/GvsCreateFilterSet.wdl

    }

-    call Utils.MergeVCFs as MergeRecalibrationFiles {
+    call PopulateFilterSetInfo as PopulateFilterSetInfoCLassic {


references will need to be updated as well obvs

Suggested change

call PopulateFilterSetInfo as PopulateFilterSetInfoCLassic {

call PopulateFilterSetInfo as PopulateFilterSetInfoClassic {

scripts/variantstore/wdl/GvsCreateFilterSet.wdl

scripts/vcf_site_level_filtering_wdl/JointVcfFiltering.wdl

src/main/java/org/broadinstitute/hellbender/tools/gvs/filtering/CreateFilteringFiles.java

gbggrant

Looks good, but I'm confused as to why vqslod is entirely null in vqsr_lite_filter_set_info.
Would like to understand prior to merging.

scripts/variantstore/wdl/GvsCreateFilterSet.wdl

samuelklee · 2023-02-01T19:55:45Z

Hi all, thanks again for working to integrate this code!

Saw some confusion in the comments above and just wanted to clarify: if you take a look at the VQSR-lite PR https://github.com/broadinstitute/gatk/pull/7954/commits that the current branch is rebased upon, you'll see that it contains a version of the Joint Genotyping WDL (which was put together by Megan for Ultima) along with Java code for the tools (which was written by me).

Both the WDL and the code have been updated in subsequent PRs. The WDL was rewritten by me in #8074; the main difference is that we no longer run SNPs and indels filtering in "series", but instead run them in a single step. However, this requires that you use the same annotations for both SNPs and indels; GVS might not be ready for that just yet, since the default WARP implementation uses different annotations. (But see also the comment here: #8074 (comment). The gist is we can easily reimplement Megan's/WARP's "serial" SNP-then-indel workflow using the simpler single-step workflow.) (EDIT: I was originally confused here, Megan’s WDL simply runs SNPs and indels separately—thanks to George for correcting me here!)

Note also that test infrastructure was moved from Travis to Github Actions between these PRs, so the Travis references above have already been cleaned up. There have also been a few additional minor PRs merged in the interim, with a couple more incoming. These PRs do not fundamentally change the interfaces of the tools/WDL, however, so I think you can update to them when you're ready.

Punchline: this branch should suffice for a first cut of a VQSR/VQSR-lite bakeoff, and although it is already slightly out of date, it shouldn't be too much work to get things updated after the first cut is done.

samuelklee and others added 20 commits October 17, 2022 13:50

Added toggle for selecting resource-matching strategies and miscellan…

78fdd8d

…eous minor fixes to new annotation-based filtering tools. (#8049)

Adding use_allele_specific_annotation arg and fixing task with empty …

c95945c

…input in JointVcfFiltering WDL (#8027) * Small changes to JointVCFFiltering WDL * making default for use_allele_specific_annotations * addressing comments

first stab

3b61d7c

wire through WDL changes

fc1c11e

Merge branch 'ah_var_store' into rsa_vqsr_lite_poc

18c33de

fixed typo

19fa6c3

set model_backend input value

971d82f

add gatk_override to JointVcfFiltering call

b888432

typo in indel_annotations

86d63c0

make model_backend optional

1b1ebf5

tabs and spaces

6b5c879

make all model_backends optional

72a9d93

use gatk 4.3.0

d525fca

no point in changing the table names as this is a POC

917d1b7

Merge branch 'ah_var_store' into rsa_vqsr_lite_poc

3c19b50

adding new branch to dockstore

7e7c74f

adding in branching logic for classic VQSR vs VQSR-Lite

b178fe5

implementing the separate schemas for the VQSR vs VQSR-Lite branches,…

95cad2e

… including Java changes necessary to produce the different tsv files

passing classic flag to indel run of CreateFilteringFiles

9c39f73

koncheto-broad requested review from rsasch and gbggrant January 12, 2023 15:05

RoriCremer reviewed Jan 17, 2023

View reviewed changes

scripts/variantstore/wdl/GvsCreateFilterSet.wdl Outdated Show resolved Hide resolved

Update GvsCreateFilterSet.wdl

3320837

cleaning up verbiage

mcovarr reviewed Jan 19, 2023

View reviewed changes

scripts/gatkcondaenv.yml.template Show resolved Hide resolved

scripts/vcf_site_level_filtering_cromwell_tests/README.md Show resolved Hide resolved

samuelklee and others added 2 commits January 23, 2023 16:13

Removed mapping error rate from estimate of denoised copy ratios outp…

bfb1a28

…ut by gCNV and updated sklearn. (#7261)

Merge branch 'ah_var_store' into VS-693_VQSR_lite

fac9ef0

rsasch approved these changes Jan 25, 2023

View reviewed changes

mcovarr reviewed Jan 26, 2023

View reviewed changes

gbggrant requested changes Jan 26, 2023

View reviewed changes

scripts/variantstore/wdl/GvsCreateFilterSet.wdl Show resolved Hide resolved

gbggrant reviewed Jan 26, 2023

View reviewed changes

scripts/variantstore/wdl/GvsCreateFilterSet.wdl Show resolved Hide resolved

gbggrant approved these changes Jan 26, 2023

View reviewed changes

cleanup up sloppy comment

4b9fa62

koncheto-broad merged commit cdb74b7 into ah_var_store Feb 2, 2023

koncheto-broad deleted the VS-693_VQSR_lite branch February 2, 2023 14:33

samuelklee mentioned this pull request Feb 2, 2023

Performed a round of ablation on new annotation-based filtering tools. #8131

Merged

This was referenced Mar 17, 2023

lb merge gvs branch #8248

Closed

testing something, please ignore #8251

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[VS-693] Add support for VQSR Lite to GvsCreateFilterSet #8157

[VS-693] Add support for VQSR Lite to GvsCreateFilterSet #8157

koncheto-broad commented Jan 12, 2023 •

edited

Loading

codecov bot commented Jan 12, 2023 •

edited

Loading

gatk-bot commented Jan 12, 2023

gatk-bot commented Jan 17, 2023

mcovarr left a comment

rsasch left a comment

mcovarr Jan 25, 2023

koncheto-broad Jan 30, 2023

mcovarr Jan 25, 2023

gbggrant left a comment

samuelklee commented Feb 1, 2023 •

edited

Loading

	call PopulateFilterSetInfo as PopulateFilterSetInfoCLassic {
	call PopulateFilterSetInfo as PopulateFilterSetInfoClassic {

[VS-693] Add support for VQSR Lite to GvsCreateFilterSet #8157

[VS-693] Add support for VQSR Lite to GvsCreateFilterSet #8157

Conversation

koncheto-broad commented Jan 12, 2023 • edited Loading

codecov bot commented Jan 12, 2023 • edited Loading

Codecov Report

gatk-bot commented Jan 12, 2023

gatk-bot commented Jan 17, 2023

mcovarr left a comment

Choose a reason for hiding this comment

rsasch left a comment

Choose a reason for hiding this comment

mcovarr Jan 25, 2023

Choose a reason for hiding this comment

koncheto-broad Jan 30, 2023

Choose a reason for hiding this comment

mcovarr Jan 25, 2023

Choose a reason for hiding this comment

gbggrant left a comment

Choose a reason for hiding this comment

samuelklee commented Feb 1, 2023 • edited Loading

koncheto-broad commented Jan 12, 2023 •

edited

Loading

codecov bot commented Jan 12, 2023 •

edited

Loading

samuelklee commented Feb 1, 2023 •

edited

Loading