v0.6.1

v0.6.1 (August 30, 2024)

Bug fixes

• Fixed the JSON schema inference warning on excluded fields; also improved error message reporting of which method triggered the error. PR #1037

v0.6.0

v0.6.0 (August 27, 2024)

Deprecations

• Deprecated WaveformExtractor usage. PR #821
• Changed the spikeinterface.tool functions (e.g. add_recording, add_sorting) to have _to_nwbfile as suffix PR #1015
• Deprecated use of compression and compression_options in VideoInterface PR #1005
• get_schema_from_method_signature has been deprecated; please use get_json_schema_from_method_signature instead. PR #1016
• neuroconv.utils.FilePathType and neuroconv.utils.FolderPathType have been deprecated; please use pydantic.FilePath and pydantic.DirectoryPath instead. PR #1017
• Changed the roiextractors.tool function (e.g. add_imaging and add_segmentation) to have the _to_nwbfile suffix [PR #1027]PR #1017

Features

• Added MedPCInterface for operant behavioral output files. PR #883
• Support SortingAnalyzer in the SpikeGLXConverterPipe. PR #821
• Added TDTFiberPhotometryInterface data interface, for converting fiber photometry data from TDT file formats. PR #920
• Add argument to add_electrodes that grants fine control of what to do with the missing values. As a side effect this drops the implicit casting to int when writing int properties to the electrodes table PR #985
• Add Plexon2 support PR #918
• Converter working with multiple VideoInterface instances PR #914
• Added helper function neuroconv.tools.data_transfers.submit_aws_batch_job for basic automated submission of AWS batch jobs. PR #384
• Data interfaces run_conversion method now performs metadata validation before running the conversion. PR #949
• Introduced null_values_for_properties to add_units_table to give user control over null values behavior PR #989

Bug fixes

• Fixed the default naming of multiple electrical series in the SpikeGLXConverterPipe. PR #957
• Write new properties to the electrode table use the global identifier channel_name, group PR #984
• Removed a bug where int64 was casted lossy to float PR #989

Improvements

• The OpenEphysBinaryRecordingInterface now uses lxml for extracting the session start time from the settings.xml file and does not depend on pyopenephys anymore. PR #971
• Swap the majority of package setup and build steps to pyproject.toml instead of setup.py. PR #955
• The DeeplabcutInterface now skips inferring timestamps from movie when timestamps are specified, running faster. PR #967
• Improve metadata writing for SpikeGLX data interface. Added contact ids, shank ids and, remove references to shanks for neuropixels 1.0. Also deprecated the previous neuroconv exclusive property "electrode_shank_number` PR #986
• Add tqdm with warning to DeepLabCut interface PR #1006
• BaseRecordingInterface now calls default metadata when metadata is not passing mimicking run_conversion behavior. PR #1012
• Added get_json_schema_from_method_signature which constructs Pydantic models automatically from the signature of any function with typical annotation types used throughout NeuroConv. PR #1016
• Replaced all interface annotations with Pydantic types. PR #1017
• Changed typehint collections (e.g. List) to standard collections (e.g. list). PR #1021
• Testing now is only one dataset per test PR #1026

v0.5.0

v0.5.0 (July 17, 2024)

Deprecations

• The usage of compression_options directly through the neuroconv.tools.audio submodule is now deprecated - users should refer to the new configure_backend method for a general approach for setting compression. PR #939
• The usage of compression and compression_opts directly through the FicTracDataInterface is now deprecated - users should refer to the new configure_backend method for a general approach for setting compression. PR #941
• The usage of compression directly through the neuroconv.tools.neo submodule is now deprecated - users should refer to the new configure_backend method for a general approach for setting compression. PR #943
• The usage of compression_options directly through the neuroconv.tools.ophys submodule is now deprecated - users should refer to the new configure_backend method for a general approach for setting compression. PR #940
• Removed the option of running interface.run_conversion without nwbfile_path argument . PR #951

Features

• Added docker image and tests for an automated Rclone configuration (with file stream passed via an environment variable). PR #902

Bug fixes

• Fixed the conversion option schema of a SpikeGLXConverter when used inside another NWBConverter. PR #922
• Fixed a case of the NeuroScopeSortingExtractor when the optional xml_file_path is not specified. PR #926
• Fixed Can't specify experiment type when converting .abf to .nwb with Neuroconv. PR #609
• Remove assumption that the ports of the Intan acquisition system correspond to electrode groupings in IntanRecordingInterface PR #933
• Add ValueError for empty metadata in make_or_load_nwbfile when an nwbfile needs to be created PR #948

Improvements

• Make annotations from the raw format available on IntanRecordingInterface. PR #934
• Add an option to suppress display the progress bar (tqdm) in VideoContext PR #937
• Automatic compression of data in the LightnignPoseDataInterface has been disabled - users should refer to the new configure_backend method for a general approach for setting compression. PR #942
• Port over dlc2nwb utility functions for ease of maintenance. PR #946

v0.4.11

v0.4.11 (June 14, 2024)

Bug fixes

• Added a skip condition in get_default_dataset_io_configurations for datasets with any zero-length axis in their full_shape. PR #894
• Added packaging explicitly to minimal requirements. PR #904
• Fixed bug when using make_or_load_nwbfile with overwrite=True on an existing (but corrupt) HDF5 file. PR #911
• Change error trigger with warning trigger when adding both OnePhotonSeries and TwoPhotonSeries to the same file (Issue #906). PR #907

Improvements

• Propagated photon_series_type to BaseImagingExtractorInterface init instead of passing it as an argument of get_metadata() and get_metadata_schema(). PR #847

v0.4.10

v0.4.10 (June 6, 2024)

Bug fixes

• Fixed bug causing overwrite of NWB GUIDE watermark. PR #890

v0.4.9

v0.4.9 (June 5, 2024)

Deprecations

• Removed stream_id as an argument from IntanRecordingInterface. PR #794
• The usage of compression and compression_opts directly through the neuroconv.tools.spikeinterface submodule are now deprecated - users should refer to the new configure_backend method for a general approach for setting compression. PR #805
• Dropped the testing of Python 3.8 on the CI. Dropped support for Python 3.8 in setup. PR #853
• Deprecated skip_features argument in add_sorting. PR #872
• Deprecate old (v1) iterator from the ecephys pipeline. PR #876

Features

• Added backend control to the make_or_load_nwbfile helper method in neuroconv.tools.nwb_helpers. PR #800
• Released the first official Docker images for the package on the GitHub Container Repository (GHCR). PR #383
• Support "one-file-per-signal" and "one-file-per-channel" mode with IntanRecordingInterface. PR #791
• Added get_default_backend_configuration method to all DataInterface classes. Also added HDF5 backend control to all standalone .run_conversion(...) methods for those interfaces. PR #801
• Added get_default_backend_configuration method to all NWBConverter classes. Also added HDF5 backend control to .run_conversion(...). PR #804
• Released the first official Docker images for the package on the GitHub Container Repository (GHCR). PR #383
• Added ScanImageMultiFileImagingInterface for multi-file (buffered) ScanImage format and changed ScanImageImagingInterface to be routing classes for single and multi-plane imaging. PR #809
• Added a function to generate ogen timestamps and data from onset times and parameters to tools.optogenetics. PR #832
• Added configure_and_write_nwbfile and optimized imports in tools.nwb_helpers module. PR #848
• configure_backend may now apply a BackendConfiguration to equivalent in-memory pynwb.NWBFile objects that have different address in RAM. PR #848
• Add support for doubled ragged arrays in add_units_table. PR #879
• Add support for doubled ragged arrays in add_electrodes. PR #881

Bug fixes

• Remove JSON Schema definitions from the properties field. PR #818
• Fixed writing waveforms directly to file. PR #799
• Avoid in-place modification of the metadata in the VideoInterface and on neo tools. PR #814
• Replaced waveform_extractor.is_extension with waveform_extractor.has_extension. PR #799
• Fixed an issue with set_aligned_starting_time for all SortingInterface's that did not have an initial segment start set (and no recording attached). PR #823
• Fixed a bug with parameterized and pytest-xdist==3.6.1 in the ScanImageImagingInterface tests. PR #829
• Added XX and XO to the base metadata schema. PR #833
• BaseImagingExtractor.add_to_nwbfile() is fixed in the case where metadata is not supplied. PR #849
• Prevent SpikeGLXConverterPipe from setting false properties on the sub-SpikeGLXNIDQInterface. PR #860
• Fixed a bug when adding ragged arrays to the electrode and units table. PR #870
• Fixed a bug where write_recording will call an empty nwbfile when passing a path. PR #877
• Fixed a bug that failed to properly include time alignment information in the output NWB file for objects added from any RecordingInterface in combination with stub_test=True. PR #884
• Fixed a bug that prevented passing nwbfile=None and a backend_configuration to NWBConverter.run_conversion. PR #885

Improvements

• Fixed docstrings related to backend configurations for various methods. PR #822
• Added automatic backend detection when a backend_configuration is passed to an interface or converter. PR #840
• Improve printing of bytes. PR #831
• Support for pathlib in source data schema validation. PR #854
• Use ZoneInfo instead of dateutil.tz in the conversion gallery. PR #858
• Exposed progress_bar_class to ecephys and ophys data iterators. PR #861
• Unified the signatures between add_units, add_sorting and write_sorting. PR #875
• Improved descriptions of all folder and file paths in the source schema, useful for rendering in the GUIDE. PR #886
• Added watermark via source_script field of NWBFile metadata. source_script_file_name is also required to be specified in this case to avoid invalidation. PR #888

Testing

• Add general test for metadata in-place modification by interfaces. PR #815

v0.4.8

v0.4.8 (March 20, 2024)

Bug fixes

• Fixed writing the electrodes field in add_electrical_series when multiple groups are present. PR #784

Improvements

• Upgraded Pydantic support to >v2.0.0. PR #767
• Absorbed the DatasetInfo model into the DatasetIOConfiguration model. PR #767
• Keyword argument field_name of the DatasetIOConfiguration.from_neurodata_object method has been renamed to dataset_name to be more consistent with its usage. This only affects direct initialization of the model; usage via the BackendConfiguration constructor and its associated helper functions in neuroconv.tools.nwb_helpers is unaffected. PR #767
• Manual construction of a DatasetIOConfiguration now requires the field dataset_name, and will be validated to match the final path of location_in_file. Usage via the automated constructors is unchanged. PR #767
• Enhance get_schema_from_method_signature to extract descriptions from the method docval. PR #771

v0.4.7

v0.4.7 (February 21, 2024)

Deprecation

• Removed .get_electrode_table_json() on the BaseRecordingExtractorInterface in favor of GUIDE specific interactions. PR #431
• Removed the SIPickleRecordingInterface and SIPickleSortingInterface interfaces. PR #757
• Removed the SpikeGLXLFPInterface interface. PR #757

Bug fixes

• LocalPathExpander matches only folder_paths or file_paths if that is indicated in the passed specification. PR #679 and [PR #675](#679
• Fixed depth consideration in partial chunking pattern for the ROI data buffer. PR #677
• Fix mapping between channel names and the electrode table when writing more than one ElectricalSeries to the NWBFile. This fixes an issue when the converter pipeline of SpikeGLXConverterPipe was writing the electrode table region of the NIDQ stream incorrectly. PR #678
• Fix configure_backend when applied to TimeSeries contents that leverage internal links for data or timestamps. PR #732

Features

• Changed the Suite2pSegmentationInterface to support multiple plane segmentation outputs. The interface now has a plane_name and channel_name arguments to determine which plane output and channel trace add to the NWBFile. PR #601
• Added create_path_template and corresponding tests PR #680
• Added tool function configure_datasets for configuring all datasets of an in-memory NWBFile to be backend specific. PR #571
• Added LightningPoseConverter to add pose estimation data and the original and the optional labeled video added as ImageSeries to NWB. PR #633
• Added gain as a required __init__ argument for TdtRecordingInterface. PR #704
• Extract session_start_time from Plexon plx recording file. PR #723

Improvements

• nwbinspector has been removed as a minimal dependency. It becomes an extra (optional) dependency with neuroconv[dandi]. PR #672
• Added a from_nwbfile class method constructor to all BackendConfiguration models. PR #673
• Added compression to FicTracDataInterface. PR #678
• Exposed block_index to all OpenEphys interfaces. PR #695
• Added support for DynamicTable columns in the configure_backend tool function. PR #700
• Refactored ScanImagingInterface to reference ROIExtractors' version of extract_extra_metadata. PR #731
• Added support for Long NHP probe types for the SpikeGLXRecorddingInterfacce. PR #701
• Remove unnecessary duplication of probe setting in SpikeGLXRecordingInterface. PR #696
• Added associated suffixes to all interfaces and converters. PR #734
• Added convenience function get_format_summaries to tools.importing (and exposed at highest level). PR #734

Testing

• RecordingExtractorInterfaceTestMixin now compares either group_name, group or a default value of ElectrodeGroup to the group property in the NWBRecordingExtractor instead of comparing group to group as it was done before PR #736
• TestScanImageImagingInterfaceRecent now checks metadata against new roiextractors implementation PR #741.

v0.4.6

v0.4.6

Back-compatability break

• Changed the metadata schema for Fluorescence and DfOverF where the traces metadata should be provided as a dict instead of a list of dicts. The name of the plane segmentation is used to determine which traces to add to the Fluorescence and DfOverF containers. PR #632

Features

• Added Pydantic data models of BackendConfiguration for both HDF5 and Zarr datasets (container/mapper of all the DatasetConfigurations for a particular file). PR #568
• Modify the filtering of traces to also filter out traces with empty values. PR #649
• Added tool function get_default_dataset_configurations for identifying and collecting all fields of an in-memory NWBFile that could become datasets on disk; and return instances of the Pydantic dataset models filled with default values for chunking/buffering/compression. PR #569
• Added tool function get_default_backend_configuration for conveniently packaging the results of get_default_dataset_configurations into an easy-to-modify mapping from locations of objects within the file to their correseponding dataset configuration options, as well as linking to a specific backend DataIO. PR #570
• Added set_probe() method to BaseRecordingExtractorInterface. PR #639

Fixes

• Fixed GenericDataChunkIterator (in hdmf.py) in the case where the number of dimensions is 1 and the size in bytes is greater than the threshold of 1 GB. PR #638
• Changed np.floor and np.prod usage to math.floor and math.prod in various files. PR #638
• Updated minimal required version of DANDI CLI; updated run_conversion_from_yaml API function and tests to be compatible with naming changes. PR #664

Improvements

• Change metadata extraction library from fparse to parse. PR #654
• The dandi CLI/API is now an optional dependency; it is still required to use the tool function for automated upload as well as the YAML-based NeuroConv CLI. PR #655

v0.4.5

v0.4.5

Back-compatibility break

• The CEDRecordingInterface has now been removed; use the Spike2RecordingInterface instead. PR #602

Features

• Added session_start_time extraction to FicTracDataInterface. PR #598
• Added imaging_plane_name keyword argument to add_imaging_plane function to determine which imaging plane to add from the metadata by name instead of imaging_plane_index.
• Added reference for imaging_plane to default plane segmentation metadata. PR #594
• Changed Compass container for Position container in the FicTracDataInterface. PR #606
• Added option to write units in meters by providing a radius in FicTracDataInterface. PR #606
• Added parent_container keyword argument to add_photon_series that defines whether to add the photon series to acquisition or 'ophys' processing module. PR #587
• Added Pydantic data models of DatasetInfo (immutable summary of core dataset values such as maximum shape and dtype) and DatasetConfiguration for both HDF5 and Zarr datasets (the optional layer that specifies chunk/buffering/compression). PR #567
• Added alignment methods to FicTracDataInterface. PR #607
• Added alignment methods support to MockRecordingInterface PR #611
• Added NeuralynxNvtInterface, which can read position tracking NVT files. PR #580
• Adding radius as a conversion factor in FicTracDataInterface. PR #619
• Coerce FicTracDataInterface original timestamps to start from 0. PR #619
• Added configuration metadata to FicTracDataInterface. PR #618
• Expose number of jobs to automatic_dandi_upload. PR #624
• Added plane_segmentation_name keyword argument to determine which plane segmentation to add from the metadata by name instead of plane_segmentation_index.
  plane_segmentation_name is exposed at BaseSegmentationExtractorInterface.add_to_nwbfile() function to support adding segmentation output from multiple planes. PR #623
• Added SegmentationImages to metadata_schema in BaseSegmentationExtractorInterface to allow for the modification of the name and description of Images container and description of the summary images. PR #622
• Default chunking pattern of RecordingInterfaces now attempts to use as many channels as possible up to 64 total, and fill with as much time as possible up to the chunk_mb. This also required raising the lower HDMF version to 3.11.0 (which introduced 10 MB default chunk sizes). PR #630

Fixes

• Remove starting_time reset to default value (0.0) when adding the rate and updating the photon_series_kwargs or roi_response_series_kwargs, in add_photon_series or add_fluorescence_traces. PR #595
• Changed the date parsing in OpenEphysLegacyRecordingInterface to datetime.strptime with the expected date format explicitly set to "%d-%b-%Y %H%M%S". PR #577
• Pin lower bound HDMF version to 3.10.0. PR #586

Deprecation

• Removed use_times and buffer_size from add_photon_series. PR #600

Testing

• Adds MockImagingInterface as a general testing mechanism for ophys imaging interfaces PR #604.