Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs for analysis of shard runtimes for balanced sharding #7645

Merged
merged 4 commits into from
Feb 14, 2022

Conversation

kcibul
Copy link
Contributor

@kcibul kcibul commented Jan 25, 2022

No description provided.

@rsasch rsasch self-assigned this Jan 26, 2022
Copy link

@rsasch rsasch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two possible changes, otherwise 👍🏻

PARAMS="includeKey=start&includeKey=end&includeKey=executionStatus&includeKey=preemptible&includeKey=shardIndex&includeKey=wdl-task-name&includeKey=jes&includeKey=jobId"

curl -sL -X 'GET' \
"https://api.firecloud.org/api/workflows/v1/${WF_ID}/metadata?${PARAMS}" \
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if you want to replace this with what I was able to get to work or just document it as an option, but I ran

"https://rawls.dsde-prod.broadinstitute.org/api/workspaces/${WORKSPACE_NAME}/submissions/${SUBMISSION_ID}/workflows/${WF_ID}?https://cromwell1-int-lb.dsde-prod.broadinstitute.org/api/workflows/v1/${WF_ID}/metadata?${PARAMS} \

where WORKSPACE_NAME is the name of the workspace including the billing project (e.g. "broad-dsde-firecloud-billing/Genomic Variant Store - GVS Quickstart") and SUBMISSION_ID is the submission ID.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought I understood this… but are you passing an entire URL after the first ??

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes...? To be completely honest, I found this API call by looking in the browser "Network" tab to see what call the "Workflow dashboard" was making; I don't think it is documented anywhere. But, since a part of the Terra UI (granted, marked as "alpha") uses it and it was the only way I was able to get any metadata for large workflows, I thought it would be helpful to write down. I would definitely understand if you don't want to add it to anything official.

@kcibul kcibul merged commit 6295133 into ah_var_store Feb 14, 2022
@kcibul kcibul deleted the kc_sharding_analysis branch February 14, 2022 20:09
This was referenced Mar 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants