Romancal 0.16.3 (24Q4_B15)

What's Changed

• only load patch table when needed by @braingram in #1367
• Populate segmentation model metadata for exposures and mosaics. by @schlafly in #1391

Full Changelog: 0.16.2...0.16.3

Romancal 0.16.2 (24Q4_B15)

What's Changed

• Add a default suffix to ELP by @nden in #1378

Full Changelog: 0.16.1...0.16.2

Romancal 0.16.1 (24Q4_B15)

What's Changed

• release metadata for 0.16.1 + update stpipe to use ModelLibrary + stcal to use outlier detection functions by @zacharyburnett in #1364

Full Changelog: 0.16.0...0.16.1

Romancal 0.16.0 (24Q4_B15)

rad 0.21.0

• Add schema for sky background
• Update schema for TVAC and FPS data

roman_datamodels 0.21.0

• Update models for sky background
• Updates for TVAC data
• Updates to improve memory usage

romancal 0.16.0

• Updates to the source catalog step
• Update tweakreg step to use source catalog
• Updates to the skymatch step
• Replace ModelContainer with ModelLibrary
• Update code to use ModelLibrary
• Updates to improve memory management

full changelog...

• remove clean_up_after_test by @braingram in #1245
• rcal-845_dsd rename highlevelpipeline to mosaicpipeline by @ddavis-stsci in #1249
• Update python from 3.9 to 311 in JenkinsfileRT_dev by @ddavis-stsci in #1251
• apply minor fixes for sonarqube by @zacharyburnett in #1250
• Refactor DQInitStep to use the RampModel method of creating ramps by @stscieisenhamer in #1258
• rcal-846 Add mosaic info to the science products RTD page by @ddavis-stsci in #1262
• add codecov_token secret to CI workflow by @zacharyburnett in #1265
• rcal-850 Addition of catalog step to mosaic pipeline by @ddavis-stsci in #1266
• Fix bug when setting meta.background in SkyMatchStep. by @mairanteodoro in #1233
• RCAL-828: add PSF bits to source catalog. by @mairanteodoro in #1243
• Fix comparison operator. by @mairanteodoro in #1269
• Rcal 854 Tweakreg step listed as skipped when run & refactor exposure pipeline by @ddavis-stsci in #1271
• pin numpy <2 by @braingram in #1275
• use resample many_to_many in outlier detection by @braingram in #1260
• replace deprecated stpipe configobj with astropy by @braingram in #1267
• RCAL-777: allow Step._datamodels_open to open association files by @braingram in #1270
• Do not set conda version in Jenkinsfile(s) by @jhunkeler in #1286
• rcal-859 Update okify script to use GA directories on artifactory by @ddavis-stsci in #1282
• add upper pin for photutils by @braingram in #1291
• unpin webbpsf by @braingram in #1288
• Rcal 860 Allow roman datamodel as input to the exposure pipeline by @ddavis-stsci in #1296
• Fix issue in constructing skycell WCS. by @schlafly in #1297
• Fix psf tests by @larrybradley in #1293
• Update exposure_pipeline.py by @mairanteodoro in #1298
• remove executable bits from non-script files by @braingram in #1299
• show logs on failed tests by @braingram in #1307
• remove pytest asdf plugin configuration for non-existant schemas by @braingram in #1305
• Remove unnecessary global variable and unit tests. by @mairanteodoro in #1314
• replace usages of copy_arrays with memmap by @zacharyburnett in #1316
• Replace ModelContainer with ModelLibrary by @braingram in #1241
• delete unreachable code by @braingram in #1322
• Pin stcal for development by @ddavis-stsci in #1326
• RCAL-830: update tweakreg to use source catalog by @mairanteodoro in #1276
• provide formatter to log_records by @braingram in #1327
• use reusable workflow from WebbPSF repository to download and cache data by @zacharyburnett in #1311
• Bump actions/labeler from 4 to 5 by @dependabot in #1231
• use monkeypatch in patch_match tests to avoid global state change by @braingram in #1319
• update labeler to v5 by @zacharyburnett in #1333
• remove creation of stray fits file by @braingram in #1338
• remove stcal upper pin by @braingram in #1340
• increase asdf minimum required version by @braingram in #1343
• Remove uses of lazy_load=False, update step input handling by @braingram in #1342
• rcal-827 Add reg test for DMS373 by @ddavis-stsci in #1348
• uncomment and use get_asn by @braingram in #1320
• RCAL-885: Updated Sky Background for Schema by @PaulHuwe in #1351
• Update absolute separation default. by @schlafly in #1352
• [SCSB-169] add user input to specify CRDS_CONTEXT by @zacharyburnett in #1361
• make crs bigger for new pars file for jump unit tests by @braingram in #1356
• RCAL-858: Create a regtest for sky match. by @mairanteodoro in #1358
• fix number of context planes by @braingram in #1355
• Populate sky metadata with zero if matching is not possible. by @schlafly in #1360

Full Changelog: 0.15.2...0.16.0

Romancal 0.15.2

What's Changed

• pin stcal>=1.7.3,<1.8.0
• metadata updates for release 0.15.2 by @zacharyburnett in #1324

Full Changelog: 0.15.1...0.15.2

Romancal 0.15.1

• roman_datamodels bug fixes
• Improved support for TVAC/FPS schemas

New Contributors

• @dependabot made their first contribution in #1230

Full Changelog: 0.15.0...0.15.1

Romancal 0.15.0 (DMS build 24Q3_B14)

RAD v 0.19.4, cumulative changes since v0.19.0

• Changed image units from e/s to DN/s.
• Add flux step schema
• Add L2 and L3 catalog schema
• Add L2 and L3 segmentation map schema

Roman Datamodels v 0.19.2, cumulative changes since v0.19.0

• Move dqflags to roman_datamodels from romancal
• Updates for FPS and TVAC schemas
• Add Bit Mask to Resample
• Add 'basic' schema to 'wfi_mosaic'
• Change image units from e/s to DN/s
• Improve handling of individual image metadata.

Romancal v 0.15.0

• The high-level processing pipeline can now resample images onto skycells
• New capability to compute skycells matching input images
• Allow DQ flags to be specified by the user in resample step
• Populate the l3 product individual_image_meta block, which tracks the metadata from input L2 images.
• Include the input L2 file names on disk in the meta.resample information
• Populate level 3 WCS keywords
• Include logs in L3 products
• Move dqflags to roman_datamodels
• Make PSF fitting the default in source detection, and improve PSF fitting.
• Resampling now populates weight maps and context images.
• Improve outlier detection.
• Add a flux calibration step to the mosaic pipeline.
• Add new a new SourceCatalog step with Webb heritage.
• Add a new test of astrometric accuracy.

General

• Updates to read-the-docs
• Updates for unit and regression tests

New Contributors

• @larrybradley made their first contribution in #1186

Full Changelog: 0.14.0...0.15.0

Romancal 0.14.0 (DMS build 24Q2_B13)

RAD v 0.19.0

• The Level 3 Mosaic schema was updated to include new metadata
• Add a wcs tag to schemas so the WCS object is now validated
• Removed unused variance schema

Roman Datamodels v 0.19.0

• Updates for metadata of the Level 3 Mosaics model
• The WCS object is now required for Level 2 and Level 3 models
• Updates to have meta.model_type to reflect current model

Romancal v 0.14.0

Level 2 products

• Activate apodized FFT interpolation in RefPix
• Accept dq array from SDF in the case of telemetry data dropouts
• Border pixels arrays are now copies and not views into the full data array. This avoids the increase of file size as with the view the original 3D data array was copied in the Level 2 products.

Level 3 products

• Update resample step to handle the L3 metadata

General

• Updates to read-the-docs
• Updates for unit and regression tests

Changelog

• Rcal 727 Add High Level Pipeline documentation, various updates by @ddavis-stsci in #1043
• RCAL-509: Update jump step docs by @WilliamJamieson in #1035
• add documentation for static preview script by @zacharyburnett in #1046
• Hotfix for regression tests broken by filename changes by @WilliamJamieson in #1050
• remove LFS tracking from .gitattributes by @zacharyburnett in #1051
• Add docs for outlier detection step. by @mairanteodoro in #1042
• Rcal 300 Updates to check for CRDS return value for each ELPP step by @ddavis-stsci in #1055
• Add unit tests and outlier module refactoring. by @mairanteodoro in #1058
• Rcal 738 Process data with at least 16 resultants (SOC-1069 & SOC-1070) by @ddavis-stsci in #1059
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #1018
• drop sphinx-asdf requirement by @braingram in #1063
• Rcal 740 Add Data Products documentation to RTD by @ddavis-stsci in #1064
• Update to outlier detection docs. by @mairanteodoro in #1062
• fix compare_asdf model difference by @braingram in #1066
• Provide a default WCS for ramp fitting. by @schlafly in #1072
• switch jenkins runs to use roman-serverless for crds by @braingram in #946
• Rcal-622 Add the ability to copy resultantdq from SDF to a rampmodel by @ddavis-stsci in #1085
• RCAL-704 Update resample step to handle the L3 meta data by @stscieisenhamer in #1057
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #1069
• Add python 3.12 devdeps testing by @WilliamJamieson in #886
• Update tweakwcs dependency by @schlafly in #1086
• RCAL-724: Add additional information to compare_asdf diff reports. by @WilliamJamieson in #1044
• Turn on the apodized FFT interpolation by @WilliamJamieson in #1017
• update requirements-sdp.txt instructions to include the sdp extra by @zacharyburnett in #1097
• Make copies of border pixel arrays when building L2 image. by @schlafly in #1098
• Fix a few incorrect dtypes used in tests by @WilliamJamieson in #1090
• metadata updates for release 0.14.0 by @jhunkeler in #1103

New Contributors

• @jhunkeler made their first contribution in #1103

Full Changelog: 0.13.0...0.14.0

Romancal 0.13.0 (DMS build 24Q1_B12)

• Ramp fitting supports uneven ramps
• Jump detection algorithm updated
• Combine jump detection and ramp fitting steps
• Support for data with truncated MA tables
• Added skymatch, outlier detection and resample steps
• Refined source catalogs in the tweakreg step
• Updates to the saturation step
• Code for the High Level Pipeline

What's Changed

• Add dev optional dependency by @WilliamJamieson in #835
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #833
• Cleanup the requirements-dev by @WilliamJamieson in #843
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #845
• Make it so we can manually trigger the CI by @WilliamJamieson in #844
• Fix get_crds_parameters for ModelContainer by @WilliamJamieson in #846
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #849
• RCAL-511 Inititial implementation of the Uneven Ramp fitting by @stscieisenhamer in #779
• Fix RTD build by @WilliamJamieson in #853
• fix opening mode for references to be read-only by @stscieisenhamer in #854
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #855
• PSF fitting methods by @bmorris3 in #794
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #860
• PSF fitting random seed fix by @bmorris3 in #862
• Remove style check from CI by @WilliamJamieson in #864
• Add NumPy ruff rule set checking by @WilliamJamieson in #863
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #873
• RCAL-643 Update roman CI/CD to use CRDS local cache by @ddavis-stsci in #857
• fix caching and outputs by @zacharyburnett in #871
• reorganize tests, make devdeps testing optional, and unpin latest Python testing by @zacharyburnett in #874
• fix optional workflow triggers by @zacharyburnett in #876
• add checkbox for regression tests to the pull request template by @zacharyburnett in #878
• Fix for failing regression tests from PSF fitting methods by @bmorris3 in #872
• use deepdiff instead of asdf.commands.diff for output and truth file comparisons by @braingram in #868
• Adjust ramps for gain, a la JWST's GainScaleStep. by @schlafly in #804
• Adjust saturation threshold depending on the read pattern by @schlafly in #836
• split requirements-dev.txt into first and third party dev dependencies by @zacharyburnett in #840
• RCAL-646 Update to install asn_from_list by @ddavis-stsci in #884
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #889
• RCAL-647 Update pipeline to include refpix step as default by @ddavis-stsci in #890
• New ramp fitting default by @schlafly in #877
• rcal-681 Update reg tests for new algorithms by @ddavis-stsci in #911
• add astropy Table support to compare_asdf by @braingram in #915
• update compare_asdf to use tolerance in more comparisons by @braingram in #917
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #921
• update url for webbpsf data in github actions CI by @braingram in #920
• use PyPI upload workflow from OpenAstronomy by @zacharyburnett in #887
• RCAL-641 Add FOV association generation by @ddavis-stsci in #931
• TweakReg regression test fix. by @mairanteodoro in #919
• add instructions for downloading WebbPSF data by @zacharyburnett in #937
• add .git to dev dependency specifications by @zacharyburnett in #932
• improve WebbPSF data caching in GitHub CI by @braingram in #923
• add webbpsf and poppy to dev requirements by @braingram in #940
• RCAL-693: Removed err array from dark tests. by @PaulHuwe in #938
• use ndarray comparison options in wcs comparison by @braingram in #941
• Rcal 504 develop the resample step by @mairanteodoro in #787
• Fix regression test artifact upload by @braingram in #950
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #930
• Filter catalog sources that are outside of the bounding box by @mcara in #947
• Update Cas22 ramp fitting to integrate with the jump detection changes in stcal by @WilliamJamieson in #933
• RCAL-513 Update uneven ramp fitting documentation by @stscieisenhamer in #944
• RCAL-644 Remove checks on CI in production code by @stscieisenhamer in #955
• Skip webbpsf tests by default by @WilliamJamieson in #958
• rcal-697 Update exposure pipeline to add tweakreg step by @ddavis-stsci in #960
• retrieve CRDS contexts in CI with a reusable workflow by @zacharyburnett in #954
• Bugfix webbpsf marker by @WilliamJamieson in #961
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #963
• RCAL-670: additional check for data quality test. by @mairanteodoro in #964
• Generate L3 exposure times from resampled exposure time images by @schlafly in #959
• rcal-700 Update exposure pipeline to correct cal_step and suffixes by @ddavis-stsci in #971
• RCAL-629: static preview generation script by @zacharyburnett in #953
• fixed asn_from_list script by @zacharyburnett in #972
• don't run tests on romancal/scripts by @zacharyburnett in #975
• Protect imports in script better by @WilliamJamieson in #977
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #974
• Source detection step option to fit model PSFs by @bmorris3 in #841
• use webbpsf data from grp in jenkins runs by @braingram in #918
• Update crds rad and roman_datamodels versions by @ddavis-stsci in #986
• RCAL-707 Update elp to replace list as input to tweakreg by @ddavis-stsci in #985
• fix issue where the path of the WebbPSF cache was not the same as the restore cache in the subsequent workflow by @zacharyburnett in #988
• fix tests_devdeps.yml workflow by @zacharyburnett in #989
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #990
• Structured array source detection catalogs by @bmorris3 in #987
• Source detection+PSF docs updates by @bmorris3 in #984
• RDM-3431: add mosaic astrometric frame test by @mairanteodoro in #993
• Remove quote indent from main header in README by @bmorris3 in #992
• minor fix for data hash in cache key by @zacharyburnett in #999
• RCAL-699: Add regression testing for new jump detection and make new jump detection default to on by @WilliamJamieson in #991
• Rcal 713 Update pyproject.toml to use stcal 1.5.0 or greater and add ref_file.crds.sw_version to the ignore list by @ddavis-stsci in #1003
• Fix issue with source catalog in TweakRe...

Romancal 0.12.0 (DMS build 23Q4_B11)

This release includes updates for association processing in the pipeline, added the reference pixel step, and some improvements in memory usage. It also includes updates for roman_datamodels for additional unit checking. The input data files now include updated quantities and the CRDS context is now at roman_0051.

What's Changed

• Update the CI by @WilliamJamieson in #706
• More CI updates by @WilliamJamieson in #709
• Fix returned result from SourceDetectionStep. by @mairanteodoro in #700
• Documentation update for rtd and the contrib by @sosey in #674
• Initial sky matching implementation by @bmorris3 in #687
• Move skymatch changelog entry to correct version by @WilliamJamieson in #713
• Replace add_ref_common with mk_ref_common by @WilliamJamieson in #712
• Add ModelContainer to romancal.datamodels. by @mairanteodoro in #710
• RCAL-590: add fit results to datamodel's meta. by @mairanteodoro in #714
• Update reference pixel docs. by @schlafly in #716
• Add a pip list step after the overrides are injected by @WilliamJamieson in #711
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #715
• add sdpdeps toxenv by @zacharyburnett in #708
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #718
• Move is_assocation to romancal by @WilliamJamieson in #719
• Fix typo in S_REGION string. by @mairanteodoro in #720
• Updates for maker_utils API changes by @WilliamJamieson in #717
• revert astropy prerelease wheel spec and use nightly Scipy wheel index by @zacharyburnett in #696
• Add regression test for TweakReg. by @mairanteodoro in #707
• update jump detection for stcal changes by @braingram in #723
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #722
• RCAL-604 update ramp fitting unit test for stcal 1.4.0 by @ddavis-stsci in #725
• RCAL-432 Update info string to use the same syntax by @ddavis-stsci in #721
• Increase code coverage for TweakReg. by @mairanteodoro in #724
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #766
• Remove myst parser from docs config by @WilliamJamieson in #773
• Add jquery sphinx to fix doc search by @WilliamJamieson in #768
• Register skymatch with stpipe by @WilliamJamieson in #770
• Use same changelog workflow as the rest of the roman repositories by @WilliamJamieson in #772
• Remove AWS install option by @WilliamJamieson in #767
• Update asdf dependency by @WilliamJamieson in #777
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #776
• remove unused extras by @zacharyburnett in #784
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #782
• Make test randomness deterministic by @WilliamJamieson in #771
• Try to avoid copying data models. by @schlafly in #774
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #788
• update build instructions in README by @zacharyburnett in #790
• RCAL-485, RCAL-558: Implement the reference pixel correction by @WilliamJamieson in #704
• BOT: Changelog check has moved by @pllim in #792
• Actually use the astropy nightly wheel. by @WilliamJamieson in #795
• rcal-606 Have source detection and tweakreg skip spectral data by @ddavis-stsci in #798
• RCAL-617 Update readme for release 11 (23Q3_B10) & placeholders for 12 (23Q4_B11) by @ddavis-stsci in #799
• pin numpy to >=1.22 by @zacharyburnett in #800
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #801
• Remove numpy 2.0 hack by @WilliamJamieson in #797
• pin sphinx_rtd_theme to newer version by @zacharyburnett in #803
• Use CRDS public server by @nden in #805
• Rcal 596 Add association processing to the pipeline code by @ddavis-stsci in #802
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #806
• RCAL-631 Update saturation logic for spectral data by @ddavis-stsci in #824
• Remove usage of pkg_resources by @WilliamJamieson in #829
• Rcal 624 regtest updates for B11, rad & roman_datamodels pin by @ddavis-stsci in #834
• rcal-627 Add metrics logger to regtests by @ddavis-stsci in #831
• metadata for 0.12.0 release by @zacharyburnett in #839

New Contributors

• @sosey made their first contribution in #674
• @schlafly made their first contribution in #716
• @pllim made their first contribution in #792

Full Changelog: 0.11.0...0.12.0