Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moving and testing ingest scripts from variantstore #6881

Merged
merged 5 commits into from
Oct 20, 2020

Conversation

meganshand
Copy link
Contributor

No description provided.

@gatk-bot
Copy link

Travis reported job failures from build 31792
Failures in the following jobs:

Test Type JDK Job ID Logs
integration openjdk11 31792.13 logs

Copy link
Contributor

@ahaessly ahaessly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a couple of questions. merge whenever

cpu: 2
}
output {
File metadata_tsv = glob("sample_*.tsv")[0]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i changed the name of the file from metadata_.tsv to sample_.tsv. it might make it more clear to update this output param. (if you do, also change it in the inputs to LoadArrays)

Int table_id
File raw_schema
File sample_list_schema
String load = "false"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is false the right default?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll change it to true.

LOAD=true
else
LOAD=false
fi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when do you not want to load?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thought was that if we end up using Google Data Transfer we need a script that will create the tables but not actually load the data. Ideally we'd add the generation of the Transfers to this script too, but I didn't get around to doing that.

Now that I look at this though, I think the code for this script is all contained within the WDL and I shouldn't have committed this extra file. I'll add a comment about the Google Data Transfer to the WDL and delete this bash script.

@meganshand meganshand merged commit 390badf into ah_var_store Oct 20, 2020
@meganshand meganshand deleted the ms_ingest_wdl branch October 20, 2020 17:59
kcibul pushed a commit that referenced this pull request Jan 29, 2021
* Copying files over from variantstore

* some changes

* forgot to turn the test back on

* fixing test

* addressing comments
kcibul pushed a commit that referenced this pull request Feb 1, 2021
* Copying files over from variantstore

* some changes

* forgot to turn the test back on

* fixing test

* addressing comments
Marianie-Simeon pushed a commit that referenced this pull request Feb 16, 2021
* Copying files over from variantstore

* some changes

* forgot to turn the test back on

* fixing test

* addressing comments
kcibul pushed a commit that referenced this pull request Mar 9, 2021
* Copying files over from variantstore

* some changes

* forgot to turn the test back on

* fixing test

* addressing comments
mmorgantaylor pushed a commit that referenced this pull request Apr 6, 2021
* Copying files over from variantstore

* some changes

* forgot to turn the test back on

* fixing test

* addressing comments
mmorgantaylor pushed a commit that referenced this pull request Apr 6, 2021
* Copying files over from variantstore

* some changes

* forgot to turn the test back on

* fixing test

* addressing comments
This was referenced Mar 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants