Correct the number of CPUs to request from Google quotas for AoU callsets #8016
Conversation
| @@ -25,7 +25,7 @@ | |||
| - **NOTE** If you want to create a large sample set after you have run the notebook, Terra provides (and recommends you use) this python [script](https://github.com/broadinstitute/firecloud-tools/tree/master/scripts/import_large_tsv) which allows you to upload a sample set to the workspace. | |||
| - **TBD This is based on VCF output; VDS output might need different increases.** For extracting VCFs as the final output for the callset, you will want to increase the Google quotas for the workspace project (you can find this in the workspace dashboard under Cloud Information > Google Project ID) to these levels (all in the workspace region): | |||
| - Persistent Disk Standard (GB): 1,000,000 GB (1 PB) | |||
| - CPUs: 64,000 | |||
| - CPUs: 6,400 | |||
There was a problem hiding this comment.
From poking around the history it looks like this has always been 64,000, though never having done a mega callset I defer to you as to which is correct. 🙂
There was a problem hiding this comment.
Fun fact: I requested 64,000 VM instances for the intermediate callset but actually got 640,000 (!)
There was a problem hiding this comment.
🤔 maybe that's the reason Google started cracking down....
| @@ -25,7 +25,7 @@ | |||
| - **NOTE** If you want to create a large sample set after you have run the notebook, Terra provides (and recommends you use) this python [script](https://github.com/broadinstitute/firecloud-tools/tree/master/scripts/import_large_tsv) which allows you to upload a sample set to the workspace. | |||
| - **TBD This is based on VCF output; VDS output might need different increases.** For extracting VCFs as the final output for the callset, you will want to increase the Google quotas for the workspace project (you can find this in the workspace dashboard under Cloud Information > Google Project ID) to these levels (all in the workspace region): | |||
| - Persistent Disk Standard (GB): 1,000,000 GB (1 PB) | |||
| - CPUs: 64,000 | |||
| - CPUs: 6,400 | |||
| - In-use IP addresses: 5,000 (this is the most challenging one and will probably require contacting the GCP account team to facilitate) | |||
| - VM instances: 64,000 | |||
There was a problem hiding this comment.
Should this change too? It might keep us off Google's naughty list and we don't use fractional CPUs anyway.
There was a problem hiding this comment.
When I checked the quotas for the Charlie workspace project, CPUs was 6,400 and VM instances was 64,000 so I kept this one the same.
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## ah_var_store #8016 +/- ##
================================================
Coverage ? 86.243%
Complexity ? 35196
================================================
Files ? 2173
Lines ? 165004
Branches ? 17792
================================================
Hits ? 142304
Misses ? 16373
Partials ? 6327 |
D'oh