-
Notifications
You must be signed in to change notification settings - Fork 587
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
read api bytes logging, upgrade bigquery client versions #7601
Conversation
src/main/java/org/broadinstitute/hellbender/tools/gvs/extract/ExtractCohort.java
Outdated
Show resolved
Hide resolved
|
||
import io.netty.util.internal.StringUtil; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is this change needed? or is it part of one of the upgrades?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We were importing the "shaded" version of this before which isn't really the library, but gradle's repackaging of it and it's not stable. This is the public version.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM — are you planning to build a jar after this is merged so we can start to use it for beta, or is this a post-beta thing?
The main purpose of this PR was to output the new estimated bytes read from the Read API for better monitoring and debugging.
However in the course of that I discovered that we were using ancient versions of the google APIs. No massive improvements from the release logs, but a lot of nice features (cleaner logging, automatic retries for certain errors, )
bigQuery 1.131.1 -> 2.3.3 Release log for bigQuery
bigQueryStorage 1.22.3 -> 2.5.0 Release log for bigQueryStorage