Releases: catalystneuro/neuroconv
Releases · catalystneuro/neuroconv
v0.6.1
v0.6.0
v0.6.0 (August 27, 2024)
Deprecations
- Deprecated
WaveformExtractor
usage. PR #821 - Changed the spikeinterface.tool functions (e.g.
add_recording
,add_sorting
) to have_to_nwbfile
as suffix PR #1015 - Deprecated use of
compression
andcompression_options
inVideoInterface
PR #1005 get_schema_from_method_signature
has been deprecated; please useget_json_schema_from_method_signature
instead. PR #1016neuroconv.utils.FilePathType
andneuroconv.utils.FolderPathType
have been deprecated; please usepydantic.FilePath
andpydantic.DirectoryPath
instead. PR #1017- Changed the roiextractors.tool function (e.g.
add_imaging
andadd_segmentation
) to have the_to_nwbfile
suffix [PR #1027]PR #1017
Features
- Added MedPCInterface for operant behavioral output files. PR #883
- Support
SortingAnalyzer
in theSpikeGLXConverterPipe
. PR #821 - Added
TDTFiberPhotometryInterface
data interface, for converting fiber photometry data from TDT file formats. PR #920 - Add argument to
add_electrodes
that grants fine control of what to do with the missing values. As a side effect this drops the implicit casting to int when writing int properties to the electrodes table PR #985 - Add Plexon2 support PR #918
- Converter working with multiple VideoInterface instances PR #914
- Added helper function
neuroconv.tools.data_transfers.submit_aws_batch_job
for basic automated submission of AWS batch jobs. PR #384 - Data interfaces
run_conversion
method now performs metadata validation before running the conversion. PR #949 - Introduced
null_values_for_properties
toadd_units_table
to give user control over null values behavior PR #989
Bug fixes
- Fixed the default naming of multiple electrical series in the
SpikeGLXConverterPipe
. PR #957 - Write new properties to the electrode table use the global identifier channel_name, group PR #984
- Removed a bug where int64 was casted lossy to float PR #989
Improvements
- The
OpenEphysBinaryRecordingInterface
now useslxml
for extracting the session start time from the settings.xml file and does not depend onpyopenephys
anymore. PR #971 - Swap the majority of package setup and build steps to
pyproject.toml
instead ofsetup.py
. PR #955 - The
DeeplabcutInterface
now skips inferring timestamps from movie when timestamps are specified, running faster. PR #967 - Improve metadata writing for SpikeGLX data interface. Added contact ids, shank ids and, remove references to shanks for neuropixels 1.0. Also deprecated the previous neuroconv exclusive property "electrode_shank_number` PR #986
- Add tqdm with warning to DeepLabCut interface PR #1006
BaseRecordingInterface
now calls default metadata when metadata is not passing mimickingrun_conversion
behavior. PR #1012- Added
get_json_schema_from_method_signature
which constructs Pydantic models automatically from the signature of any function with typical annotation types used throughout NeuroConv. PR #1016 - Replaced all interface annotations with Pydantic types. PR #1017
- Changed typehint collections (e.g.
List
) to standard collections (e.g.list
). PR #1021 - Testing now is only one dataset per test PR #1026
v0.5.0
v0.5.0 (July 17, 2024)
Deprecations
- The usage of
compression_options
directly through theneuroconv.tools.audio
submodule is now deprecated - users should refer to the newconfigure_backend
method for a general approach for setting compression. PR #939 - The usage of
compression
andcompression_opts
directly through theFicTracDataInterface
is now deprecated - users should refer to the newconfigure_backend
method for a general approach for setting compression. PR #941 - The usage of
compression
directly through theneuroconv.tools.neo
submodule is now deprecated - users should refer to the newconfigure_backend
method for a general approach for setting compression. PR #943 - The usage of
compression_options
directly through theneuroconv.tools.ophys
submodule is now deprecated - users should refer to the newconfigure_backend
method for a general approach for setting compression. PR #940 - Removed the option of running
interface.run_conversion
withoutnwbfile_path
argument . PR #951
Features
- Added docker image and tests for an automated Rclone configuration (with file stream passed via an environment variable). PR #902
Bug fixes
- Fixed the conversion option schema of a
SpikeGLXConverter
when used inside anotherNWBConverter
. PR #922 - Fixed a case of the
NeuroScopeSortingExtractor
when the optionalxml_file_path
is not specified. PR #926 - Fixed
Can't specify experiment type when converting .abf to .nwb with Neuroconv
. PR #609 - Remove assumption that the ports of the Intan acquisition system correspond to electrode groupings in
IntanRecordingInterface
PR #933 - Add ValueError for empty metadata in
make_or_load_nwbfile
when an nwbfile needs to be created PR #948
Improvements
- Make annotations from the raw format available on
IntanRecordingInterface
. PR #934 - Add an option to suppress display the progress bar (tqdm) in
VideoContext
PR #937 - Automatic compression of data in the
LightnignPoseDataInterface
has been disabled - users should refer to the newconfigure_backend
method for a general approach for setting compression. PR #942 - Port over
dlc2nwb
utility functions for ease of maintenance. PR #946
v0.4.11
v0.4.11 (June 14, 2024)
Bug fixes
- Added a skip condition in
get_default_dataset_io_configurations
for datasets with any zero-length axis in theirfull_shape
. PR #894 - Added
packaging
explicitly to minimal requirements. PR #904 - Fixed bug when using
make_or_load_nwbfile
withoverwrite=True
on an existing (but corrupt) HDF5 file. PR #911 - Change error trigger with warning trigger when adding both
OnePhotonSeries
andTwoPhotonSeries
to the same file (Issue #906). PR #907
Improvements
- Propagated
photon_series_type
toBaseImagingExtractorInterface
init instead of passing it as an argument ofget_metadata()
andget_metadata_schema()
. PR #847
v0.4.10
v0.4.9
v0.4.9 (June 5, 2024)
Deprecations
- Removed
stream_id
as an argument fromIntanRecordingInterface
. PR #794 - The usage of
compression
andcompression_opts
directly through theneuroconv.tools.spikeinterface
submodule are now deprecated - users should refer to the newconfigure_backend
method for a general approach for setting compression. PR #805 - Dropped the testing of Python 3.8 on the CI. Dropped support for Python 3.8 in setup. PR #853
- Deprecated skip_features argument in
add_sorting
. PR #872 - Deprecate old (v1) iterator from the ecephys pipeline. PR #876
Features
- Added
backend
control to themake_or_load_nwbfile
helper method inneuroconv.tools.nwb_helpers
. PR #800 - Released the first official Docker images for the package on the GitHub Container Repository (GHCR). PR #383
- Support "one-file-per-signal" and "one-file-per-channel" mode with
IntanRecordingInterface
. PR #791 - Added
get_default_backend_configuration
method to allDataInterface
classes. Also added HDF5backend
control to all standalone.run_conversion(...)
methods for those interfaces. PR #801 - Added
get_default_backend_configuration
method to allNWBConverter
classes. Also added HDF5backend
control to.run_conversion(...)
. PR #804 - Released the first official Docker images for the package on the GitHub Container Repository (GHCR). PR #383
- Added
ScanImageMultiFileImagingInterface
for multi-file (buffered) ScanImage format and changedScanImageImagingInterface
to be routing classes for single and multi-plane imaging. PR #809 - Added a function to generate ogen timestamps and data from onset times and parameters to
tools.optogenetics
. PR #832 - Added
configure_and_write_nwbfile
and optimized imports intools.nwb_helpers
module. PR #848 configure_backend
may now apply aBackendConfiguration
to equivalent in-memorypynwb.NWBFile
objects that have different address in RAM. PR #848- Add support for doubled ragged arrays in
add_units_table
. PR #879 - Add support for doubled ragged arrays in
add_electrodes
. PR #881
Bug fixes
- Remove JSON Schema
definitions
from theproperties
field. PR #818 - Fixed writing waveforms directly to file. PR #799
- Avoid in-place modification of the metadata in the
VideoInterface
and on neo tools. PR #814 - Replaced
waveform_extractor.is_extension
withwaveform_extractor.has_extension
. PR #799 - Fixed an issue with
set_aligned_starting_time
for allSortingInterface
's that did not have an initial segment start set (and no recording attached). PR #823 - Fixed a bug with
parameterized
andpytest-xdist==3.6.1
in theScanImageImagingInterface
tests. PR #829 - Added
XX
andXO
to the base metadata schema. PR #833 BaseImagingExtractor.add_to_nwbfile()
is fixed in the case where metadata is not supplied. PR #849- Prevent
SpikeGLXConverterPipe
from setting false properties on the sub-SpikeGLXNIDQInterface
. PR #860 - Fixed a bug when adding ragged arrays to the electrode and units table. PR #870
- Fixed a bug where
write_recording
will call an empty nwbfile when passing a path. PR #877 - Fixed a bug that failed to properly include time alignment information in the output NWB file for objects added from any
RecordingInterface
in combination withstub_test=True
. PR #884 - Fixed a bug that prevented passing
nwbfile=None
and abackend_configuration
toNWBConverter.run_conversion
. PR #885
Improvements
- Fixed docstrings related to backend configurations for various methods. PR #822
- Added automatic
backend
detection when abackend_configuration
is passed to an interface or converter. PR #840 - Improve printing of bytes. PR #831
- Support for pathlib in source data schema validation. PR #854
- Use
ZoneInfo
instead ofdateutil.tz
in the conversion gallery. PR #858 - Exposed
progress_bar_class
to ecephys and ophys data iterators. PR #861 - Unified the signatures between
add_units
,add_sorting
andwrite_sorting
. PR #875 - Improved descriptions of all folder and file paths in the source schema, useful for rendering in the GUIDE. PR #886
- Added watermark via
source_script
field ofNWBFile
metadata.source_script_file_name
is also required to be specified in this case to avoid invalidation. PR #888
Testing
- Add general test for metadata in-place modification by interfaces. PR #815
v0.4.8
v0.4.8 (March 20, 2024)
Bug fixes
- Fixed writing the
electrodes
field inadd_electrical_series
when multiple groups are present. PR #784
Improvements
- Upgraded Pydantic support to
>v2.0.0
. PR #767 - Absorbed the
DatasetInfo
model into theDatasetIOConfiguration
model. PR #767 - Keyword argument
field_name
of theDatasetIOConfiguration.from_neurodata_object
method has been renamed todataset_name
to be more consistent with its usage. This only affects direct initialization of the model; usage via theBackendConfiguration
constructor and its associated helper functions inneuroconv.tools.nwb_helpers
is unaffected. PR #767 - Manual construction of a
DatasetIOConfiguration
now requires the fielddataset_name
, and will be validated to match the final path oflocation_in_file
. Usage via the automated constructors is unchanged. PR #767 - Enhance
get_schema_from_method_signature
to extract descriptions from the method docval. PR #771
v0.4.7
v0.4.7 (February 21, 2024)
Deprecation
- Removed
.get_electrode_table_json()
on theBaseRecordingExtractorInterface
in favor of GUIDE specific interactions. PR #431 - Removed the
SIPickleRecordingInterface
andSIPickleSortingInterface
interfaces. PR #757 - Removed the
SpikeGLXLFPInterface
interface. PR #757
Bug fixes
- LocalPathExpander matches only
folder_paths
orfile_paths
if that is indicated in the passed specification. PR #679 and [PR #675](#679 - Fixed depth consideration in partial chunking pattern for the ROI data buffer. PR #677
- Fix mapping between channel names and the electrode table when writing more than one
ElectricalSeries
to the NWBFile. This fixes an issue when the converter pipeline ofSpikeGLXConverterPipe
was writing the electrode table region of the NIDQ stream incorrectly. PR #678 - Fix
configure_backend
when applied toTimeSeries
contents that leverage internal links fordata
ortimestamps
. PR #732
Features
- Changed the
Suite2pSegmentationInterface
to support multiple plane segmentation outputs. The interface now has aplane_name
andchannel_name
arguments to determine which plane output and channel trace add to the NWBFile. PR #601 - Added
create_path_template
and corresponding tests PR #680 - Added tool function
configure_datasets
for configuring all datasets of an in-memoryNWBFile
to be backend specific. PR #571 - Added
LightningPoseConverter
to add pose estimation data and the original and the optional labeled video added as ImageSeries to NWB. PR #633 - Added gain as a required
__init__
argument forTdtRecordingInterface
. PR #704 - Extract session_start_time from Plexon plx recording file. PR #723
Improvements
nwbinspector
has been removed as a minimal dependency. It becomes an extra (optional) dependency withneuroconv[dandi]
. PR #672- Added a
from_nwbfile
class method constructor to allBackendConfiguration
models. PR #673 - Added compression to
FicTracDataInterface
. PR #678 - Exposed
block_index
to all OpenEphys interfaces. PR #695 - Added support for
DynamicTable
columns in theconfigure_backend
tool function. PR #700 - Refactored
ScanImagingInterface
to reference ROIExtractors' version ofextract_extra_metadata
. PR #731 - Added support for Long NHP probe types for the
SpikeGLXRecorddingInterfacce
. PR #701 - Remove unnecessary duplication of probe setting in
SpikeGLXRecordingInterface
. PR #696 - Added associated suffixes to all interfaces and converters. PR #734
- Added convenience function
get_format_summaries
totools.importing
(and exposed at highest level). PR #734
Testing
RecordingExtractorInterfaceTestMixin
now compares eithergroup_name
,group
or a default value ofElectrodeGroup
to thegroup
property in theNWBRecordingExtractor
instead of comparinggroup
togroup
as it was done before PR #736TestScanImageImagingInterfaceRecent
now checks metadata against new roiextractors implementation PR #741.
v0.4.6
v0.4.6
Back-compatability break
- Changed the metadata schema for
Fluorescence
andDfOverF
where the traces metadata should be provided as a dict instead of a list of dicts. The name of the plane segmentation is used to determine which traces to add to theFluorescence
andDfOverF
containers. PR #632
Features
- Added Pydantic data models of
BackendConfiguration
for both HDF5 and Zarr datasets (container/mapper of all theDatasetConfiguration
s for a particular file). PR #568 - Modify the filtering of traces to also filter out traces with empty values. PR #649
- Added tool function
get_default_dataset_configurations
for identifying and collecting all fields of an in-memoryNWBFile
that could become datasets on disk; and return instances of the Pydantic dataset models filled with default values for chunking/buffering/compression. PR #569 - Added tool function
get_default_backend_configuration
for conveniently packaging the results ofget_default_dataset_configurations
into an easy-to-modify mapping from locations of objects within the file to their correseponding dataset configuration options, as well as linking to a specific backend DataIO. PR #570 - Added
set_probe()
method toBaseRecordingExtractorInterface
. PR #639
Fixes
- Fixed GenericDataChunkIterator (in hdmf.py) in the case where the number of dimensions is 1 and the size in bytes is greater than the threshold of 1 GB. PR #638
- Changed
np.floor
andnp.prod
usage tomath.floor
andmath.prod
in various files. PR #638 - Updated minimal required version of DANDI CLI; updated
run_conversion_from_yaml
API function and tests to be compatible with naming changes. PR #664
Improvements
v0.4.5
v0.4.5
Back-compatibility break
- The
CEDRecordingInterface
has now been removed; use theSpike2RecordingInterface
instead. PR #602
Features
- Added
session_start_time
extraction toFicTracDataInterface
. PR #598 - Added
imaging_plane_name
keyword argument toadd_imaging_plane
function to determine which imaging plane to add from the metadata by name instead ofimaging_plane_index
. - Added reference for
imaging_plane
to default plane segmentation metadata. PR #594 - Changed Compass container for Position container in the
FicTracDataInterface
. PR #606 - Added option to write units in meters by providing a radius in
FicTracDataInterface
. PR #606 - Added
parent_container
keyword argument toadd_photon_series
that defines whether to add the photon series to acquisition or 'ophys' processing module. PR #587 - Added Pydantic data models of
DatasetInfo
(immutable summary of core dataset values such as maximum shape and dtype) andDatasetConfiguration
for both HDF5 and Zarr datasets (the optional layer that specifies chunk/buffering/compression). PR #567 - Added alignment methods to
FicTracDataInterface
. PR #607 - Added alignment methods support to
MockRecordingInterface
PR #611 - Added
NeuralynxNvtInterface
, which can read position tracking NVT files. PR #580 - Adding radius as a conversion factor in
FicTracDataInterface
. PR #619 - Coerce
FicTracDataInterface
original timestamps to start from 0. PR #619 - Added configuration metadata to
FicTracDataInterface
. PR #618 - Expose number of jobs to
automatic_dandi_upload
. PR #624 - Added
plane_segmentation_name
keyword argument to determine which plane segmentation to add from the metadata by name instead ofplane_segmentation_index
.
plane_segmentation_name
is exposed atBaseSegmentationExtractorInterface.add_to_nwbfile()
function to support adding segmentation output from multiple planes. PR #623 - Added
SegmentationImages
to metadata_schema inBaseSegmentationExtractorInterface
to allow for the modification of the name and description of Images container and description of the summary images. PR #622 - Default chunking pattern of RecordingInterfaces now attempts to use as many channels as possible up to 64 total, and fill with as much time as possible up to the
chunk_mb
. This also required raising the lower HDMF version to 3.11.0 (which introduced 10 MB default chunk sizes). PR #630
Fixes
- Remove
starting_time
reset to default value (0.0) when adding the rate and updating thephoton_series_kwargs
orroi_response_series_kwargs
, inadd_photon_series
oradd_fluorescence_traces
. PR #595 - Changed the date parsing in
OpenEphysLegacyRecordingInterface
todatetime.strptime
with the expected date format explicitly set to"%d-%b-%Y %H%M%S"
. PR #577 - Pin lower bound HDMF version to
3.10.0
. PR #586
Deprecation
- Removed
use_times
andbuffer_size
fromadd_photon_series
. PR #600
Testing
- Adds
MockImagingInterface
as a general testing mechanism for ophys imaging interfaces PR #604.