Debug pipeline job serialization #167
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bbimber
added a commit
that referenced
this pull request
Aug 24, 2022
Drop GATK3 and JBrowse from sequence_tools_install
bbimber
added a commit
that referenced
this pull request
Aug 24, 2022
* Support direct CRAM conversion in alignment pipelines * Remove jboss and standardize gradle files with develop * Switch error to warning * Update GenotypeGVCFHandler to include non-variant sites when making sites-only VCF * Update case in toLower * More informative error message * Fix argument in SamtoolsCramConverter * Update jbrowse dependencies (#165) * Add CRAM to allowable JBrowse track types * Initial support for pbmm2 and pbsv * Initial support for vulcan long read aligner * Support for quality metrics from nimble * Ensure output directory exists * Fix to JBrowse 2 CRAM tracks * Add additional vulcan alignment outputs * Fix filepath typo * Update nimble alignment defaults * Add debug message for nimble * Bugfix to nimble metrics import when running as alignment * Improve column width for nimble panels * Make nimble max_hits_to_report configurable * Skip merge unaligned for long-read aligners * Add UCell calculation step * Better handling for job resume after nimble failure * Add validation and bugfix for Nimble metrics import * Improve log messages for nimble metrics import * Support maxGenotypeCount for GenotypeGVCFs * Bump terser from 5.12.1 to 5.14.2 in /jbrowse (#166) Bumps [terser](https://github.com/terser/terser) from 5.12.1 to 5.14.2. - [Release notes](https://github.com/terser/terser/releases) - [Changelog](https://github.com/terser/terser/blob/master/CHANGELOG.md) - [Commits](https://github.com/terser/terser/commits) --- updated-dependencies: - dependency-name: terser dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bugfix to the order of nimble gz and error checks * Bugfix to the order of nimble gz and error checks * Add debugging * Add debugging to cDNA prep * Add view to assist in management of single-cell data * Add another view to assist in management of single-cell data * Remove no longer needed install of jbrowse 1 * Update report filter * Allow batch assignment of sequence libraries to runs by lane * Add new param to fail 10x processing if too few cells are found * Support nimble strandedness filter * Remove no-longer-needed nimble handler * Support CellMembrane and Seurat IntegrateData * Update SplitSeurat minCellsToKeep to allow fractions * Register new IntegrateData step * Support RIRA CalculateGeneComponentScores * Bugfix to RIRA CalculateGeneComponentScores * Skip GATK annotations to avoid broadinstitute/gatk#7938 * Return to using MS- prefix for cell hashing dual-index barcodes * Add additional single-cell filters and support ReblockGVCF * Allow celltypist model to run across genomes * Fix nextclade syntax * Fix nextclade syntax * Fix nextclade syntax * Fix nextclade syntax * Support GATK ReblockGVCF * Switch queries to use POST * Bugfix to ReblockGvcfHandler * Add admin action to manually update URI on ExpData objects * Allow GenotypeGVCFHandler to create genomicsdb workspaces on-the-fly * Ensure file is cached for GenotypeGVCFs exclude_intervals * More specific regex * Serialize SequenceAnalysisJobSupport outside of PipelineJob to reduce the size of the job's JSON * Allow deserialization of legacy JSON files containing support property * Update artifactory URLs * Improve unit test * Debug pipeline job serialization (#167) * Debug pipeline job serialization * Fix bug with gene scores not being saved * Increase RAM for remote FASTQC jobs * Test fixes * Only serialize SequenceJobSupport to disk when running on webserver * More fixes around serialization of SequenceJobSupport * Reduce HaplotypeCaller max-alternate-alleles * Support additional GenotypeGVCF params * Fix tests * Fix value for createsSeuratObjects on several steps * Improve warning messages * Improve warning messages * Improve warning messages * Improve GenotypeGVCFs logging * Improve GenotypeGVCFs logging and drop old params * Prior to GenotypeGVCFs, create workspaces with padding over the provided intervals * Refactor VcfComparisonStep to support VCF output(s), and add mGAP-release-specific version * Add validation for SamtoolsCramConverter * Reduce the amount of serialization to disk from SequenceJobSupport * Prepare sequence pipeline client code for non-savable params * Allow Seurat merge object name to be excluded from saved templates * Allow GenotypeGVCFs to locally cache support files * Update picard syntax to match upcoming argument changes * Further reduce sequence support serialization * Add creation of bgzipped genomes to standard genome import * Bugfix to genome gzipping * Update picard version for tests * Bugfix to alignment and skipping merge unaligned reads * Match picard version to sequence tests * Correct picard version * Improve logging Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.