Enable call caching of TSV generation in GvsImportGenomes

[mmorgantaylor]

this PR:

• changes CreateVariantIngestFiles to name the output files in a predictable way - i.e. rather than using a sample_id, it uses the name of the input gvcf. e.g. pet_001_NA12878.tsv becomes pet_001_NA12878.haplotypeCalls.reblocked.vcf.gz.tsv

  • added a test in CreateVariantIngestFilesIntegrationTest to assert that the files are named as expected

• changes the GvsImportGenomes.wdl to:

  • check whether, for the given input gvcf file and for each of pet, vet, and sample_info, the output TSV already exists somewhere in the output directory. it checks subdirectories.
  • if the output TSV exists in a set_X subdirectory, we move that file back into the parent directory so that subsetting works as desired when we get to LoadTables
  • if the output TSV exists in a done subdirectory, we exit with an error

notes:

• this does not check whether the sample is in the same table_id (e.g. pet_001 versus pet_002)

this has been tested as follows:

• ran once with an exit 1 before bq load, to simulate generating TSVs and putting them into set_X subdirectories and then exiting, simulating a permissions or other bq issue
• removed LOCKFILE, removed exit before bq load, then ran again - TSVs were not regenerated, the existing ones were moved into the parent directory and loaded properly into bq
• then ran again with the same samples - as expected, errored out because the TSVs already existed in a done folder

[ahaessly]

LGTM!

[kcibul]

Looks great -- I think this pushes the limits of what we should do in bash (for future maintainers) but does the trick.