Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

close StorageAPIAvroReader appropriately #7583

Merged
merged 1 commit into from
Nov 25, 2021
Merged

Conversation

kcibul
Copy link
Contributor

@kcibul kcibul commented Nov 24, 2021

When running with multiple table (ie _001, _002) this would show up in the logs due to the bigquery client not being closed. It didn't cause any harm, but is alarming! Closing the ranges-based readers appropriately now.

Nov 23, 2021 5:45:53 AM io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference cleanQueue
SEVERE: *~*~*~ Channel ManagedChannelImpl{logId=121, target=bigquerystorage.googleapis.com:443} was not shutdown properly!!! ~*~*~*
    Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true.
java.lang.RuntimeException: ManagedChannel allocation site
	at io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference.<init>(ManagedChannelOrphanWrapper.java:93)
	at io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:53)
	at io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:44)
	at io.grpc.internal.ManagedChannelImplBuilder.build(ManagedChannelImplBuilder.java:615)
	at io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:261)
	at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:360)
	at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.access$1800(InstantiatingGrpcChannelProvider.java:81)
	at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider$1.createSingleChannel(InstantiatingGrpcChannelProvider.java:231)
	at com.google.api.gax.grpc.ChannelPool.create(ChannelPool.java:72)
	at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:241)
	at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:219)
	at com.google.api.gax.rpc.ClientContext.create(ClientContext.java:199)
	at com.google.cloud.bigquery.storage.v1.stub.EnhancedBigQueryReadStub.create(EnhancedBigQueryReadStub.java:89)
	at com.google.cloud.bigquery.storage.v1.BigQueryReadClient.<init>(BigQueryReadClient.java:129)
	at com.google.cloud.bigquery.storage.v1.BigQueryReadClient.create(BigQueryReadClient.java:110)
	at com.google.cloud.bigquery.storage.v1.BigQueryReadClient.create(BigQueryReadClient.java:102)
	at org.broadinstitute.hellbender.utils.bigquery.StorageAPIAvroReader.<init>(StorageAPIAvroReader.java:60)
	at org.broadinstitute.hellbender.tools.gvs.extract.ExtractCohortEngine.createSortedReferenceRangeCollectionFromBigQuery(ExtractCohortEngine.java:851)
	at org.broadinstitute.hellbender.tools.gvs.extract.ExtractCohortEngine.createVariantsFromUnsortedBigQueryRanges(ExtractCohortEngine.java:891)
	at org.broadinstitute.hellbender.tools.gvs.extract.ExtractCohortEngine.traverse(ExtractCohortEngine.java:224)
	at org.broadinstitute.hellbender.tools.gvs.extract.ExtractCohort.traverse(ExtractCohort.java:335)
	at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1058)
	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:140)
	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)
	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)
	at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
	at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
	at org.broadinstitute.hellbender.Main.main(Main.java:289)

@kcibul kcibul requested a review from rsasch November 24, 2021 21:12
Copy link

@rsasch rsasch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the quick fix 👍🏻 (assuming tests pass)

@kcibul kcibul merged commit 44aee08 into ah_var_store Nov 25, 2021
@kcibul kcibul deleted the kc_close_reader branch November 25, 2021 16:08
This was referenced Mar 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants