Mixed types for neurodata_type_inc
when defining NWBGroupSpec
#37
-
Hi, I'm learning about creating extension and I want to know how to define mixed types in the Minimal exampleStep 1 - Create extension:from pynwb.spec import NWBNamespaceBuilder, NWBGroupSpec, NWBAttributeSpec
from pynwb.core import NWBDataInterface
from hdmf.common import DynamicTable, VectorData
name = 'raw_acquisition_module'
ns_path = name + ".namespace.yaml"
ext_source = name + ".extensions.yaml"
ns_builder = NWBNamespaceBuilder(name + ' extensions', name, version='0.1.0')
ns_builder.include_type('NWBDataInterface', namespace='core')
ns_builder.include_type('DynamicTable', namespace='hdmf-common')
rawacqmod = NWBGroupSpec(
neurodata_type_def='RawAcquisitionModule',
neurodata_type_inc='NWBDataInterface',
attributes=[
NWBAttributeSpec(name='name', doc='Name of this acquisition module', dtype='text'),
NWBAttributeSpec(name='description', doc='Description of this acquisition module', dtype='text'),
],
doc='Module/container to add acqusition raw data into',
quantity='*',
default_name='UntitledRawAcquisitionModule',
groups=[
NWBGroupSpec(
neurodata_type_inc='NWBDataInterface', <------- this is where I have to want mixed types
doc='NWBDataInterface',
quantity='*',
),
]
)
ns_builder.add_spec(ext_source, rawacqmod)
ns_builder.export(ns_path) It's the "arrow-ed" line that I want to add both Step 2 - Import and register class# In a different file from Step 1
import pandas as pd
import numpy as np
from datetime import datetime
from dateutil import tz
from pynwb import NWBFile, TimeSeries, NWBHDF5IO
from pynwb import register_class, load_namespaces, get_class
from pynwb.file import MultiContainerInterface
from pynwb.core import NWBDataInterface
from hdmf.common import DynamicTable, VectorData
from hdmf.utils import docval, popargs_to_dict, get_docval, popargs
ns_path = 'config/raw_acquisition_module.namespace.yaml'
load_namespaces(ns_path)
@register_class('RawAcquisitionModule', 'raw_acquisition_module')
class RawAcquisitionModule(MultiContainerInterface):
__nwbfields__ = ('name', 'description')
__clsconf__ = [
{
'attr': 'nwb_data_interfaces',
'add': 'add',
'type': (NWBDataInterface, DynamicTable),
'get': 'get'
}
]
# I have to manually define these again because to have the ability to `add`, for some reason,
# I'd have to define `__clsconf__` and that somehow erases the attributes defined in the original namespace
@docval(
{
'name': 'name',
'doc': 'Name of this acquisition module',
'type': str,
'default': 'UntitledRawAcquisitionModule'
},
{
'name': 'description',
'doc': 'Description of this acquisition module',
'type': str,
'default': None
},
{
'name': 'nwb_data_interfaces',
'doc': 'NWBDataInterface',
'type': (list, tuple, dict, NWBDataInterface, DynamicTable), <------- this is where I have to declare explicitly 2 types
'default': None
}
)
def __init__(self, **kwargs):
description, nwb_data_interfaces = popargs("description", "nwb_data_interfaces", kwargs)
super().__init__(**kwargs)
self.description = description
self.nwb_data_interfaces = nwb_data_interfaces Step 3 - Usage# In same file as Step 2
nwbfile = NWBFile(
session_description='abc',
identifier='xyz',
session_start_time= datetime(2018, 1, 2, 3, 4, 5, tzinfo=tz.gettz("US/Pacific"))
)
ts = TimeSeries(
name = 'raw_series',
description = 'raw series',
data = np.random.randint(3,size=10),
timestamps = np.arange(10).astype('float'),
unit = '-'
)
tbl = DynamicTable(
name = 'id_lookup',
description = 'lookup table',
columns = [
VectorData(name = 'the_id', description='id', data=[0,1,2]),
VectorData(name = 'the_name', description='name', data=['a','b','c'])
]
)
mod = RawAcquisitionModule(
name = 'raw_mod',
description = 'raw module'
)
mod.add(ts)
mod.add(tbl)
nwbfile.add_acquisition(mod)
print(f'In acquisition: {nwbfile.acquisition}') This is what came out of In acquisition: {'raw_mod': raw_mod __main__.RawAcquisitionModule at 0x140313029216048
Fields:
description: raw module
nwb_data_interfaces: {
id_lookup <class 'hdmf.common.table.DynamicTable'>, <---- this is here when creating
raw_series <class 'pynwb.base.TimeSeries'>
}
} Then save: with NWBHDF5IO('test.nwb', mode='w') as io:
io.write(nwbfile) Step 4 - Open and compareWhen I try to open the file, only the series is there, the table is gone: with NWBHDF5IO('test.nwb', mode='r', load_namespaces=True) as io:
nwbfile_reload = io.read()
print(nwbfile_reload.acquisition)
-------
{'raw_mod': raw_mod __main__.RawAcquisitionModule at 0x140311279003744
Fields:
description: raw module
nwb_data_interfaces: {
raw_series <class 'pynwb.base.TimeSeries'> <------ only series remain, table disappears
}
} System
|
Beta Was this translation helpful? Give feedback.
Replies: 5 comments 7 replies
-
@tuanpham96 I'm curious what your use-case is that you want acquisition modules. We considered building that into the standard but didn't have many compelling use-cases. Your code is very close. I would try this: from pynwb.spec import NWBNamespaceBuilder, NWBGroupSpec, NWBAttributeSpec
from pynwb.core import NWBDataInterface
from hdmf.common import DynamicTable, VectorData
name = 'ndx-acquisition-module'
ns_path = name + ".namespace.yaml"
ext_source = name + ".extensions.yaml"
ns_builder = NWBNamespaceBuilder(name + ' extensions', name, version='0.1.0')
ns_builder.include_type('NWBDataInterface', namespace='core')
ns_builder.include_type('DynamicTable', namespace='hdmf-common')
rawacqmod = NWBGroupSpec(
neurodata_type_def='AcquisitionModule',
neurodata_type_inc='NWBDataInterface',
doc='Module/container to add acquisition raw data into',
quantity='*',
groups=[
NWBGroupSpec(
neurodata_type_inc='NWBDataInterface',
doc='NWBDataInterface',
quantity='*',
),
NWBGroupSpec(
neurodata_type_inc='DynamicTable',
doc='DynamicTable',
quantity='*',
),
]
)
ns_builder.add_spec(ext_source, rawacqmod)
ns_builder.export(ns_path)
You might be able to just use |
Beta Was this translation helpful? Give feedback.
-
NWBAttributeSpec(name='description', doc='Description of this acquisition module', dtype='text'),
mod = AcquisitionModule(
name = 'raw_mod',
# description = 'raw module',
nwb_data_interfaces = dict(), # both dict() and []
dynamic_tables = dict()
) seems like a bug.
|
Beta Was this translation helpful? Give feedback.
-
OK so this issue: mod = AcquisitionModule(
name = 'raw_mod',
# description = 'raw module',
nwb_data_interfaces = dict(), # both dict() and []
dynamic_tables = dict()
) was resolved 2 weeks ago in the dev branch of hdmf: hdmf-dev/hdmf#779 |
Beta Was this translation helpful? Give feedback.
-
For both attempts below, rawacqmod = NWBGroupSpec(
neurodata_type_def='AcquisitionModule',
neurodata_type_inc='NWBDataInterface',
doc='Module/container to add acquisition raw data into',
quantity='*',
attributes = [
NWBAttributeSpec(name='description', doc='Description of this acquisition module', dtype='text'),
],
groups=[
NWBGroupSpec(
neurodata_type_inc='NWBDataInterface',
doc='NWBDataInterface',
quantity='*',
),
NWBGroupSpec(
neurodata_type_inc='DynamicTable',
doc='DynamicTable',
quantity='*',
),
]
) Attempt 1: Defining
|
Beta Was this translation helpful? Give feedback.
-
Yes, this is exactly what I was thinking, but it seems like doing this with the mapper fits your use-case better. Q1: we'll have to discuss the other questions. We'll get back to you asap, but with SfN it might be a little while. |
Beta Was this translation helpful? Give feedback.
For both attempts below,
description
is put back toNWBGroupSec(attributes=...)
when creating the extension, no need to definename
. Usinghdmf 3.4.6.post0.dev4
.