Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: #527

Open
2 tasks done
jc192015 opened this issue Jul 27, 2023 · 40 comments
Open
2 tasks done

[Bug]: #527

jc192015 opened this issue Jul 27, 2023 · 40 comments
Assignees
@lawrence-mbf

Comments

@jc192015
Copy link

What happened?

When running NWB validation for a created nwb file, I got the following error message indicating that "Could not construct TimeIntervals object due to: columns must be the same length. " I checked the length of columns (all elements in 'colnames' ) are the same. Could you please let me know how to fix that?
Thank you in advance for your time.

Steps to Reproduce

nwb.intervals_trials = types.core.TimeIntervals(...
            'colnames', [data.trialTypeStr; data.trialPropertiesHash.keyNames .';{'start_time'; 'stop_time'}],... %stop_time will be determined later
            'description', 'trial data and properties', ...
            'start_time', intervals_trials_start_time,...
            'stop_time', types.core.VectorData('data', endTimestamps,'description', 'the end time of each trial'), ...
            'id', intervals_trials_id,...
            'timeseries', types.core.VectorData('data', cell2mat(trial_timeseries),'description', 'A group of timeseries'),...
            'timeseries_index', types.core.VectorIndex('data', cumsum(ts_len),'target', types.untyped.ObjectView('/intervals/trials/timeseries')));

        for i=1:length(data.trialTypeStr)
            nwb.intervals_trials.vectordata.set(data.trialTypeStr{i}, ...
                types.core.VectorData('data', data.trialTypeMat(i,:),...
                'description', data.trialTypeStr{i}));
        end

        for i=1:length(data.trialPropertiesHash.keyNames)
            descr = data.trialPropertiesHash.descr{i};
            if iscellstr(descr)
                descr = strjoin(descr, newline);
            end
            nwb.intervals_trials.vectordata.set(data.trialPropertiesHash.keyNames{i}, ...
                types.core.VectorData(...
                'data', data.trialPropertiesHash.value{i}, ...
                'description', descr));
        end

Error Message

'Could not construct TimeIntervals object due to: columns must be the same length'

Operating System

Windows

Matlab Version

R2023a

Code of Conduct
@lawrence-mbf lawrence-mbf self-assigned this Jul 27, 2023
@lawrence-mbf
Copy link
Collaborator

Hi @jc192015 is the id column also matching in size? That's considered part of the table even if it's not in colnames

@jc192015
Copy link
Author

jc192015 commented Jul 27, 2023 via email

@lawrence-mbf lawrence-mbf mentioned this issue Jul 27, 2023
Closed
2 tasks
@lawrence-mbf
Copy link
Collaborator

@jc192015 Is the data in each VectorIndex column the same length as id? When you have VectorIndex columns their lengths are factored into the size of the table, instead of the base column.

timeseries_index for instance should have the same number of elements as id. However, timeseries does not have to and can be much larger.

@lawrence-mbf
Copy link
Collaborator

@jc192015 Can you also post the full stack of the error?

@jc192015
Copy link
Author

%%%%%%%%%%% Validation results %%%%%%%%%

(base) C:\Windows\System32>nwbinspector D:\Data_share\NWB\ --config dandi
C:\ProgramData\Anaconda3\lib\site-packages\hdmf\backends\hdf5\h5tools.py:211: UserWarning: No cached namespaces found in D:\Data_share\NWB\BAYLORNL23_20190320.nwb
warnings.warn(msg)
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit1SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit2SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit3SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit4SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit5SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit6SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit7SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit8CS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit8SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1256, in construct
obj = self.new_container(cls, builder.source, parent, builder.attributes.get(self.__spec.id_key()),
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1269, in new_container
obj.init(**kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\pynwb\epoch.py", line 32, in init
super().init(**kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\common\table.py", line 361, in init
raise ValueError("columns must be the same length")
ValueError: columns must be the same length

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\ProgramData\Anaconda3\Scripts\nwbinspector.exe_main.py", line 7, in
File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 1157, in call
return self.main(*args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 783, in invoke
return _callback(*args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\nwbinspector\nwbinspector.py", line 271, in inspect_all_cli
messages = list(
File "C:\ProgramData\Anaconda3\lib\site-packages\nwbinspector\nwbinspector.py", line 403, in inspect_all
nwbfile = robust_s3_read(io.read)
File "C:\ProgramData\Anaconda3\lib\site-packages\nwbinspector\utils.py", line 173, in robust_s3_read
raise exc
File "C:\ProgramData\Anaconda3\lib\site-packages\nwbinspector\utils.py", line 168, in robust_s3_read
return command(*command_args, **command_kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\pynwb_init.py", line 304, in read
file = super().read(**kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\backends\hdf5\h5tools.py", line 477, in read
return super().read(**kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\backends\io.py", line 60, in read
container = self.__manager.construct(f_builder)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\manager.py", line 284, in construct
result = self.__type_map.construct(builder, self, None)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\manager.py", line 795, in construct
return obj_mapper.construct(builder, build_manager, parent)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1226, in construct
subspecs = self.__get_subspec_values(builder, self.spec, manager)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1155, in __get_subspec_values
self.__get_sub_builders(groups, spec.groups, manager, ret)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1206, in __get_sub_builders
ret.update(self.__get_subspec_values(sub_builder, subspec, manager))
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1155, in __get_subspec_values
self.__get_sub_builders(groups, spec.groups, manager, ret)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1208, in __get_sub_builders
ret[subspec] = manager.construct(sub_builder)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\manager.py", line 280, in construct
result = self.__type_map.construct(builder, self, parent)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\manager.py", line 795, in construct
return obj_mapper.construct(builder, build_manager, parent)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1260, in construct
raise ConstructError(builder, msg) from ex
hdmf.build.errors.ConstructError: (root/intervals/trials GroupBuilder {'attributes': {'colnames': array(['HitR', 'HitL', 'ErrR', 'ErrL', 'NoLickR', 'NoLickL', 'LickEarly',
'StimTrials', 'PoleInTime', 'PoleOutTime', 'CueTime', 'GoodTrials',
'PhotostimulationType', 'start_time', 'stop_time'], dtype=object), 'description': 'trial data and properties', 'help': 'A general epoch object', 'namespace': 'core', 'neurodata_type': 'TimeIntervals'}, 'groups': {}, 'datasets': {'CueTime': root/intervals/trials/CueTime DatasetBuilder {'attributes': {'description': 'Start of Response period (in seconds, relative to trial start time)', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'ErrL': root/intervals/trials/ErrL DatasetBuilder {'attributes': {'description': 'ErrL', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'ErrR': root/intervals/trials/ErrR DatasetBuilder {'attributes': {'description': 'ErrR', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'GoodTrials': root/intervals/trials/GoodTrials DatasetBuilder {'attributes': {'description': '0 for trials in which mice are not performing, discard for analyses', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'HitL': root/intervals/trials/HitL DatasetBuilder {'attributes': {'description': 'HitL', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'HitR': root/intervals/trials/HitR DatasetBuilder {'attributes': {'description': 'HitR', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'LickEarly': root/intervals/trials/LickEarly DatasetBuilder {'attributes': {'description': 'LickEarly', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'NoLickL': root/intervals/trials/NoLickL DatasetBuilder {'attributes': {'description': 'NoLickL', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'NoLickR': root/intervals/trials/NoLickR DatasetBuilder {'attributes': {'description': 'NoLickR', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'PhotostimulationType': root/intervals/trials/PhotostimulationType DatasetBuilder {'attributes': {'description': '"0"--non-stimulation trials;\n"1"--photoinhibition of left ALM; One light spot (diameter 1 mm); Early Delay (start from the onset of delay epoch); 800ms (200ms ramp);\n"2"--photoinhibition of right ALM; One light spot (diameter 1 mm); Early Delay (start from the onset of delay epoch); 800ms (200ms ramp);\n"3"--photoinhibition of both left and right ALM; Four light spots for each side; Early Delay (start from the onset of delay epoch); 800ms (200ms ramp);\n"NaN and others"--discard (stimulation configuration for other purposes, should not analyze)', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'PoleInTime': root/intervals/trials/PoleInTime DatasetBuilder {'attributes': {'description': 'Start of Sample period (in seconds, relative to trial start time)', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'PoleOutTime': root/intervals/trials/PoleOutTime DatasetBuilder {'attributes': {'description': 'Start of Delay period (in seconds, relative to trial start time)', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'StimTrials': root/intervals/trials/StimTrials DatasetBuilder {'attributes': {'description': 'StimTrials', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'id': root/intervals/trials/id DatasetBuilder {'attributes': {'help': 'unique identifiers for a list of elements', 'namespace': 'core', 'neurodata_type': 'ElementIdentifiers'}, 'data': }, 'start_time': root/intervals/trials/start_time DatasetBuilder {'attributes': {'description': 'the start time of each trial', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'stop_time': root/intervals/trials/stop_time DatasetBuilder {'attributes': {'description': 'the end time of each trial', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'timeseries': root/intervals/trials/timeseries DatasetBuilder {'attributes': {'description': 'A group of timeseries', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'TimeSeriesReferenceVectorData'}, 'data': <hdmf.backends.hdf5.h5_utils.BuilderH5TableDataset object at 0x000001527A9E26A0>}, 'timeseries_index': root/intervals/trials/timeseries_index DatasetBuilder {'attributes': {'help': 'indexes into a list of values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorIndex', 'target': root/intervals/trials/timeseries DatasetBuilder {'attributes': {'description': 'A group of timeseries', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'TimeSeriesReferenceVectorData'}, 'data': <hdmf.backends.hdf5.h5_utils.BuilderH5TableDataset object at 0x000001527A9E26A0>}}, 'data': }}, 'links': {}}, 'Could not construct TimeIntervals object due to: columns must be the same length')

%%%%%%%%% Validation ended, look at the last sentence, 'Could not construct TimeIntervals object due to: columns must be the same length' %%%%%%%%%

@lawrence-mbf
Copy link
Collaborator

Would it be possible to share either this NWB file or alternatively a minimal script that produces a NWB file which also fails the nwbinspector?

@lawrence-mbf
Copy link
Collaborator

Furthermore, what version of the schema are you currently using? VectorData should be in hdmf_common if you're writing a new file with a new schema.

@jc192015
Copy link
Author

How can I share the NWB file with you?

@lawrence-mbf
Copy link
Collaborator

Is there a more secure way we could communicate? I can share a onedrive link that you can upload your file to but I will need at least an email address to share it with. Either the one you're currently using or a burner one you can use publicly.

@jc192015
Copy link
Author

[email protected]

@lawrence-mbf
Copy link
Collaborator

Sent an invite, please check your inbox and spam folder. If you have not received it in a while please let me know and we'll have to figure out another way.

@jc192015
Copy link
Author

uploaded. Thank you in advance.

@lawrence-mbf
Copy link
Collaborator

Try transposing the data provided for the column CueTime. nwbinspector (and by extension pynwb and DANDI) prefer C-style row-major ordering of their data whereas MATLAB prefers Fortran-style column-major ordering so when you're using multi-dimensional data like CueTime make sure that the last dimension matches as that is what is used for height checks in the table.

Note that for QOL reasons, vectors for the most part do not care what dimension they are on so you do not need to change those values in the table.

@jc192015
Copy link
Author

After transposing "CueTime" , I got the following outcome.

%%%%%%%%%% Validation %%%%%%%%%

(base) C:\Windows\System32>nwbinspector D:\Data_share\NWB\ --config dandi
C:\ProgramData\Anaconda3\lib\site-packages\hdmf\backends\hdf5\h5tools.py:211: UserWarning: No cached namespaces found in D:\Data_share\NWB\BAYLORNL23_20190320.nwb
warnings.warn(msg)
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit1SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit2SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit3SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit4SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit5SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit6SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit7SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit8CS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit8SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
Inspecting NWBFiles...: 0%| | 0/1 [00:00<?, ?it/s]C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit1SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit2SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit3SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit4SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit5SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit6SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit7SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit8CS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit8SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "

NWBInspector Report Summary

Timestamp: 2023-07-27 15:20:24.448653-05:00
Platform: Windows-10-10.0.19045-SP0
NWBInspector version: 0.4.29

Found 53 issues over 1 files:
10 - PYNWB_VALIDATION
19 - CRITICAL
10 - BEST_PRACTICE_VIOLATION
14 - BEST_PRACTICE_SUGGESTION

0 PYNWB_VALIDATION

0.0 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: root/general/experimenter - 'None' object at location 'general/experimenter'
Message: incorrect shape - expected an array of shape '[None]', got non-array data 'Jia Zhu'

0.1 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: root/general/related_publications - 'None' object at location 'general/related_publications'
Message: incorrect shape - expected an array of shape '[None]', got non-array data 'Jia Zhu, *, Hana Hasanbegovi�, *, Liu D. Liu, Zhenyu Gao, #, Nuo Li, #Activity map of a cortico-cerebellar loop underlying motor planning2023, Nature Neuroscience'

0.2 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: VectorIndex/description - 'None' object at location 'intervals/trials/timeseries_index'
Message: argument missing

0.3 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: VectorIndex - 'None' object at location 'intervals/trials/timeseries_index'
Message: incorrect type - expected 'uint8', got 'float64'

0.4 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: VectorIndex/description - 'None' object at location 'units/spike_times_index'
Message: argument missing

0.5 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: VectorIndex - 'None' object at location 'units/spike_times_index'
Message: incorrect type - expected 'uint8', got 'float64'

0.6 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: Units/waveforms/unit - 'None' object at location 'units/waveforms'
Message: argument missing

0.7 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: Units/waveforms - 'None' object at location 'units/waveforms'
Message: incorrect type - expected 'numeric', got 'object'

0.8 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: Units/waveforms - 'None' object at location 'units/waveforms'
Message: incorrect shape - expected '[None, None]', got '(9,)'

0.9 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: VectorData/description - 'None' object at location 'units/trials_index'
Message: argument missing

1 CRITICAL

1.10 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit8SS'
Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.

1.11 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit8CS'
Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.

1.12 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit7SS'
Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.

1.13 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit6SS'
Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.

1.14 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit5SS'
Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.

1.15 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit4SS'
Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.

1.16 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit3SS'
Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.

1.17 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit2SS'
Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.

1.18 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_electrical_series_dims - 'SpikeEventSeries' object at location '/analysis/unit1SS'
Message: The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.

1.19 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_subject_age - 'Subject' object at location '/general/subject'
Message: Subject age, '20180503', does not follow ISO 8601 duration format, e.g. 'P2Y' for 2 years or 'P23W' for 23 weeks. You may also specify a range using a '/' separator, e.g., 'P1D/P3D' for an age range somewhere from 1 to 3 days. If you cannot specify the upper bound of the range, you may leave the right side blank, e.g., 'P90Y/' means 90 years old or older.

1.20 D:\Data_share\NWB\BAYLORNL23_20190320.nwb and 8 other files: check_dynamic_table_region_data_validity - 'DynamicTableRegion' object with name 'electrodes'
Message: Some elements of electrodes are out of range because they are greater than the length of the target table. Note that data should contain indices, not ids.

2 BEST_PRACTICE_VIOLATION

2.21 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'OptogeneticSeries' object at location '/stimulus/presentation/laser_power'
Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.

2.22 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit8SS'
Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.

2.23 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit8CS'
Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.

2.24 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit7SS'
Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.

2.25 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit6SS'
Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.

2.26 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit5SS'
Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.

2.27 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit4SS'
Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.

2.28 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit3SS'
Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.

2.29 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit2SS'
Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.

2.30 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_resolution - 'SpikeEventSeries' object at location '/analysis/unit1SS'
Message: 'resolution' should use -1.0 or NaN for unknown instead of 0.0.

3 BEST_PRACTICE_SUGGESTION

3.31 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_description - 'Device' object at location '/general/devices/laser-473nm (Utralaser)'
Message: Description is missing.

3.32 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_description - 'Device' object at location '/general/devices/Cambridge NeuroTech 64ch H2 probe 2x32 shank1:ch1-32 shank2:ch33-64 (Cambridge NeuroTech)'
Message: Description is missing.

3.33 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_small_dataset_compression - 'OptogeneticSeries' object at location '/stimulus/presentation/laser_power'
Message: data is not compressed. Consider enabling compression when writing a dataset.

3.34 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_small_dataset_compression - 'SpikeEventSeries' object at location '/analysis/unit8SS'
Message: data is not compressed. Consider enabling compression when writing a dataset.

3.35 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_small_dataset_compression - 'SpikeEventSeries' object at location '/analysis/unit6SS'
Message: data is not compressed. Consider enabling compression when writing a dataset.

3.36 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_experimenter_form - 'NWBFile' object at location '/'
Message: The name of experimenter 'Jia Zhu' does not match any of the accepted DANDI forms: 'LastName, Firstname', 'LastName, FirstName MiddleInitial.' or 'LastName, FirstName, MiddleName'.

3.37 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_doi_publications - 'NWBFile' object at location '/'
Message: Metadata /general/related_publications 'Jia Zhu, *, Hana Hasanbegovi�, *, Liu D. Liu, Zhenyu Gao, #, Nuo Li, #Activity map of a cortico-cerebellar loop underlying motor planning2023, Nature Neuroscience' does not start with 'doi: ###' and is not an external 'doi' link.

3.38 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_column_binary_capability - 'TimeIntervals' object with name 'trials'
Message: Column 'HitR' uses 'floats' but has binary values [0. 1.]. Consider making it boolean instead and renaming the column to start with 'is_'; doing so will save 1.49KB.

3.39 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_column_binary_capability - 'TimeIntervals' object with name 'trials'
Message: Column 'HitL' uses 'floats' but has binary values [0. 1.]. Consider making it boolean instead and renaming the column to start with 'is_'; doing so will save 1.49KB.

3.40 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_column_binary_capability - 'TimeIntervals' object with name 'trials'
Message: Column 'ErrR' uses 'floats' but has binary values [0. 1.]. Consider making it boolean instead and renaming the column to start with 'is_'; doing so will save 1.49KB.

3.41 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_column_binary_capability - 'TimeIntervals' object with name 'trials'
Message: Column 'ErrL' uses 'floats' but has binary values [0. 1.]. Consider making it boolean instead and renaming the column to start with 'is_'; doing so will save 1.49KB.

3.42 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_column_binary_capability - 'TimeIntervals' object with name 'trials'
Message: Column 'NoLickR' uses 'floats' but has binary values [0. 1.]. Consider making it boolean instead and renaming the column to start with 'is_'; doing so will save 1.49KB.

3.43 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_column_binary_capability - 'TimeIntervals' object with name 'trials'
Message: Column 'LickEarly' uses 'floats' but has binary values [0. 1.]. Consider making it boolean instead and renaming the column to start with 'is_'; doing so will save 1.49KB.

3.44 D:\Data_share\NWB\BAYLORNL23_20190320.nwb: check_column_binary_capability - 'TimeIntervals' object with name 'trials'
Message: Column 'GoodTrials' uses 'integers' but has binary values [0 1]. Consider making it boolean instead and renaming the column to start with 'is_'; doing so will save 639.00B.

%%%%%%%%%%%% CRITICAL label appears %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%% Fake upload %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

(base) C:\Windows\System32\000572>dandi organize D:\Data_share\NWB\ -f dry
2023-07-27 15:23:17,785 [ INFO] Note: NumExpr detected 40 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
2023-07-27 15:23:24,395 [ INFO] NumExpr defaulting to 8 threads.
2023-07-27 15:23:26,576 [ INFO] Loading metadata from 1 files
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 40 concurrent workers.
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit1SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit2SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit3SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit4SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit5SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit6SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit7SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit8CS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py:86: UserWarning: SpikeEventSeries 'unit8SS': The second dimension of data does not match the length of electrodes, but instead the first does. Data is oriented incorrectly and should be transposed.
warnings.warn("%s '%s': The second dimension of data does not match the length of electrodes, "
[Parallel(n_jobs=-1)]: Done 1 tasks | elapsed: 12.4s
[Parallel(n_jobs=-1)]: Done 1 out of 1 | elapsed: 12.4s finished
DRY: D:\Data_share\NWB\BAYLORNL23_20190320.nwb -> sub-BAYLORNL23\sub-BAYLORNL23_ecephys+ogen.nwb
2023-07-27 15:23:39,131 [ INFO] Organized 0 out of 1 paths. Visit C:\Windows\System32\000572/
2023-07-27 15:23:39,131 [ INFO] Logs saved in C:\Users\NuoLiLab\AppData\Local\dandi\dandi-cli\Logs\20230727202316Z-21988.log

@jc192015
Copy link
Author

After transposing the data related to "unitXXX, " I got the following outcome.

%%%%%%%%%%%% Validation %%%%%%%%%%%%%%%%%%%%%

(base) C:\Windows\System32>nwbinspector D:\Data_share\NWB\ --config dandi
C:\ProgramData\Anaconda3\lib\site-packages\hdmf\backends\hdf5\h5tools.py:211: UserWarning: No cached namespaces found in D:\Data_share\NWB\BAYLORNL23_20190320T.nwb
warnings.warn(msg)
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1256, in construct
obj = self.new_container(cls, builder.source, parent, builder.attributes.get(self.__spec.id_key()),
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1269, in new_container
obj.init(**kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\pynwb\ecephys.py", line 125, in init
raise ValueError('Must provide the same number of timestamps and spike events')
ValueError: Must provide the same number of timestamps and spike events

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\ProgramData\Anaconda3\Scripts\nwbinspector.exe_main.py", line 7, in
File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 1157, in call
return self.main(*args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 783, in invoke
return _callback(*args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\nwbinspector\nwbinspector.py", line 271, in inspect_all_cli
messages = list(
File "C:\ProgramData\Anaconda3\lib\site-packages\nwbinspector\nwbinspector.py", line 403, in inspect_all
nwbfile = robust_s3_read(io.read)
File "C:\ProgramData\Anaconda3\lib\site-packages\nwbinspector\utils.py", line 173, in robust_s3_read
raise exc
File "C:\ProgramData\Anaconda3\lib\site-packages\nwbinspector\utils.py", line 168, in robust_s3_read
return command(*command_args, **command_kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\pynwb_init.py", line 304, in read
file = super().read(**kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\backends\hdf5\h5tools.py", line 477, in read
return super().read(**kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\backends\io.py", line 60, in read
container = self.__manager.construct(f_builder)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\manager.py", line 284, in construct
result = self.__type_map.construct(builder, self, None)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\manager.py", line 795, in construct
return obj_mapper.construct(builder, build_manager, parent)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1226, in construct
subspecs = self.__get_subspec_values(builder, self.spec, manager)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1155, in __get_subspec_values
self.__get_sub_builders(groups, spec.groups, manager, ret)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1206, in __get_sub_builders
ret.update(self.__get_subspec_values(sub_builder, subspec, manager))
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1155, in __get_subspec_values
self.__get_sub_builders(groups, spec.groups, manager, ret)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1198, in __get_sub_builders
sub_builder = self.__flatten(sub_builder, subspec, manager)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1211, in __flatten
tmp = [manager.construct(b) for b in sub_builder]
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1211, in
tmp = [manager.construct(b) for b in sub_builder]
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\manager.py", line 280, in construct
result = self.__type_map.construct(builder, self, parent)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\manager.py", line 795, in construct
return obj_mapper.construct(builder, build_manager, parent)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\utils.py", line 644, in func_call
return func(args[0], **pargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\hdmf\build\objectmapper.py", line 1260, in construct
raise ConstructError(builder, msg) from ex
hdmf.build.errors.ConstructError: (root/analysis/unit1SS GroupBuilder {'attributes': {'comments': 'cellType: PCSimpleSpikes', 'description': 'single unit1SS with cell type information and approximate depth', 'help': 'Snapshots of spike events from data.', 'namespace': 'core', 'neurodata_type': 'SpikeEventSeries'}, 'groups': {}, 'datasets': {'control': root/analysis/unit1SS/control DatasetBuilder {'attributes': {}, 'data': }, 'control_description': root/analysis/unit1SS/control_description DatasetBuilder {'attributes': {}, 'data': }, 'data': root/analysis/unit1SS/data DatasetBuilder {'attributes': {'conversion': 1.0, 'resolution': 0.0, 'unit': 'volt'}, 'data': }, 'electrodes': root/analysis/unit1SS/electrodes DatasetBuilder {'attributes': {'description': 'Electrodes involved with these spike events', 'help': 'a subset (i.e. slice or region) of a DynamicTable', 'namespace': 'core', 'neurodata_type': 'DynamicTableRegion', 'table': root/general/extracellular_ephys/electrodes GroupBuilder {'attributes': {'colnames': array(['x', 'y', 'z', 'imp', 'location', 'filtering', 'group',
'group_name'], dtype=object), 'description': 'Electrodes', 'help': 'A column-centric table', 'namespace': 'core', 'neurodata_type': 'DynamicTable'}, 'groups': {}, 'datasets': {'filtering': root/general/extracellular_ephys/electrodes/filtering DatasetBuilder {'attributes': {'description': 'description of hardware filtering', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'group': root/general/extracellular_ephys/electrodes/group DatasetBuilder {'attributes': {'description': 'a reference to the ElectrodeGroup this electrode is a part of', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': <hdmf.backends.hdf5.h5_utils.BuilderH5ReferenceDataset object at 0x0000026EF93A3AC0>}, 'group_name': root/general/extracellular_ephys/electrodes/group_name DatasetBuilder {'attributes': {'description': 'the name of the ElectrodeGroup this electrode is a part of', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'id': root/general/extracellular_ephys/electrodes/id DatasetBuilder {'attributes': {'help': 'unique identifiers for a list of elements', 'namespace': 'core', 'neurodata_type': 'ElementIdentifiers'}, 'data': }, 'imp': root/general/extracellular_ephys/electrodes/imp DatasetBuilder {'attributes': {'description': 'the impedance of the channel', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'location': root/general/extracellular_ephys/electrodes/location DatasetBuilder {'attributes': {'description': 'the location of channel within the subject e.g. brain region', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'x': root/general/extracellular_ephys/electrodes/x DatasetBuilder {'attributes': {'description': 'the x coordinate of the channel location', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'y': root/general/extracellular_ephys/electrodes/y DatasetBuilder {'attributes': {'description': 'the y coordinate of the channel location', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }, 'z': root/general/extracellular_ephys/electrodes/z DatasetBuilder {'attributes': {'description': 'the z coordinate of the channel location', 'help': 'Values for a list of elements', 'namespace': 'core', 'neurodata_type': 'VectorData'}, 'data': }}, 'links': {}}}, 'data': }, 'timestamps': root/analysis/unit1SS/timestamps DatasetBuilder {'attributes': {'interval': 1, 'unit': 'second'}, 'data': }}, 'links': {}}, 'Could not construct SpikeEventSeries object due to: Must provide the same number of timestamps and spike events')

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Fake update %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

(base) C:\Windows\System32\000572>dandi organize D:\Data_share\NWB\ -f dry
2023-07-27 15:34:02,754 [ INFO] Note: NumExpr detected 40 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
2023-07-27 15:34:02,754 [ INFO] NumExpr defaulting to 8 threads.
2023-07-27 15:34:04,826 [ INFO] Loading metadata from 1 files
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 40 concurrent workers.
[Parallel(n_jobs=-1)]: Done 1 tasks | elapsed: 4.2s
[Parallel(n_jobs=-1)]: Done 1 out of 1 | elapsed: 4.2s finished
2023-07-27 15:34:09,182 [ WARNING] Completely empty record for D:\Data_share\NWB\BAYLORNL23_20190320T.nwb
2023-07-27 15:34:09,183 [ INFO] Logs saved in C:\Users\NuoLiLab\AppData\Local\dandi\dandi-cli\Logs\20230727203401Z-14636.log
Error: 1 out of 1 files were found not containing all necessary metadata: D:\Data_share\NWB\BAYLORNL23_20190320T.nwb

@lawrence-mbf
Copy link
Collaborator

@jc192015 Is this dataset already on DANDI or are you trying to upload it do DANDI?

As far as I can tell, the Spike Event Series shapes look right but I'll need to investigate this further. The error you're getting is also a bug in pynwb.

@jc192015
Copy link
Author

Yes, I am struggling to upload nwbs to DANDI.

@lawrence-mbf
Copy link
Collaborator

I see, I will need to warn you that DANDIsets need to be at least schema version 2.1 or higher. The file you've created is still version 2.0b, you will most likely need to adapt the script to at least schema version 2.1. This may also impact why the nwbinspector is failing your file.

@lawrence-mbf
Copy link
Collaborator

Note we are still looking at it, the error still indicates an shape error which I have not been able to validate.

@jc192015
Copy link
Author

I don't know how to modify the following code to meet MatNWB 2.6.0 requirment. That is why I am still using 2.0b.

trials_epoch = types.core.TimeIntervals(...
'start_time', intervals_trials_start_time,...
'colnames', [data.trialTypeStr; data.trialPropertiesHash.keyNames .';...
{'start_time'; 'stop_time'}],...
'description', 'trial data and properties', ...
'stop_time', intervals_trials_stop_time,...
'id', intervals_trials_id_data,...
'timeseries', intervals_trials_timeseries,...
'timeseries_index', intervals_trials_timeseries_index);

@oruebel
Copy link
Contributor

oruebel commented Jul 28, 2023

trials_epoch = types.core.TimeIntervals(...

I believe the main change for TimeIntervals is that the timeseries columns has changed to the TimeSeriesReferenceVectorData type instead of using the base VectorData. Other than changing the name of the type, the definition for that column is still the same.

https://nwb-schema.readthedocs.io/en/latest/format.html?highlight=TimeIntervals#sec-timeseriesreferencevectordata

@lawrence-mbf
Copy link
Collaborator

@jc192015 my suggestion would be to at least upgrade the schema to 2.1.0 using the MAtNWB version that you have. I then recommend working with the file using the newest release of MatNWB since that supports all schema versions after 2.1.0 (the MatNWB version only specifies the most recent supported schema).

@jc192015
Copy link
Author

I generated a new nwb file using MatNWB 2.6.0.2. But I got an error message indicating that "Could not construct Units object due to: columns must be the same length." But I have checked the "Units" . The length of "id", "soike_times_index", and "waveforms" are the same. I uploaded related files "BAYLORNL23_20190320-2.6.0.2.nwb" and "Message for file made on 20230728_NL23_20190320-2.6.0.2.txt" to the onedrive that you invited me yesterday for your reference.
Thank you.

@jc192015
Copy link
Author

jc192015 commented Jul 28, 2023 via email

@lawrence-mbf
Copy link
Collaborator

There is a column called trials in the Units table. If it is supposed to be there you will need a VectorIndex named trials_index with 9 elements. We do have trials-specific tables so my assumption is that it's not supposed to be there and should perhaps be in the intervals_trials table or elsewhere.

@jc192015
Copy link
Author

jc192015 commented Jul 28, 2023 via email

@lawrence-mbf
Copy link
Collaborator

In 2.6.0 all VectorIndex objects are valid VectorData objects so you should add trials_index to the vectordata property in the Units table instead. In fact, this should be done for all VectorIndex objects created in the script.

@jc192015
Copy link
Author

jc192015 commented Jul 28, 2023 via email

@lawrence-mbf
Copy link
Collaborator

Try this:

nwb.units = types.core.Units('colnames',...
    {'spike_times', 'trials', 'waveforms'},...
    'description', 'Analysed Spike Events');

unitTrials = types.hdmf_common.VectorData(...
    'description', 'A large group of trial IDs for each unit',...
    'data', []);

trials_idx = types.hdmf_common.VectorIndex(...
    'data', [],...
    'target', types.untyped.ObjectView('/units/trials'));

nwb.units.vectordata.set('trials_index', trials_idx);
nwb.units.vectordata.set('trials', unitTrials);

Also note that you should remove any vector index columns from colnames. They, along with the vectordata they are referencing, consitute a single column in the Units table (and all Dynamic Table objects).

@jc192015
Copy link
Author

jc192015 commented Jul 28, 2023 via email

@lawrence-mbf
Copy link
Collaborator

It looks like the file is displayed properly so I don't think so. Could you share that exported file in onedrive I can confirm if that's true or not?

@jc192015
Copy link
Author

jc192015 commented Jul 28, 2023 via email

@lawrence-mbf
Copy link
Collaborator

I don't have any issues reading this file. That error that you're seeing might be from either the script or some other thing you've set up to run after reading the file.

@jc192015
Copy link
Author

jc192015 commented Jul 28, 2023 via email

@lawrence-mbf
Copy link
Collaborator

Oh that's interesting. I guess the second dimension is now the one that is checked for valid spike event series. Can you transpose the spike event series data? If you did before generating this file then undo that transpose.

@jc192015
Copy link
Author

jc192015 commented Jul 28, 2023 via email

@lawrence-mbf
Copy link
Collaborator

Very strange. Could you share this version with the onedrive so we can take a look at it?

@jc192015
Copy link
Author

jc192015 commented Jul 28, 2023 via email

@lawrence-mbf lawrence-mbf mentioned this issue Jul 28, 2023
Open
3 tasks
@lawrence-mbf
Copy link
Collaborator

@jc192015 I have opened a new issue here for discussion regarding this error: NeurodataWithoutBorders/pynwb#1746

Hopefully this is easily resolvable.

@jc192015
Copy link
Author

jc192015 commented Jul 28, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lawrence-mbf lawrence-mbf

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jc192015 @oruebel @lawrence-mbf