Re-introduce ElectrodesTable to store properties of the electrodes in place

[h-mayorquin]

This is a re-statement on #3 which ended up being derailed by the discussion of the AnatomicalCoordinatesTable and the specific Neuropixels case that was brought up. I want to make the case again here to re-focus the discussion.

In the current schema we are making a distinction between Probe and ProbeModel as the difference between the device in the experimental setup and the device in the probe catalogue as you would get from a manufacturer (altought see #7 for some gray areas).

Another possible distinction to make along those lines is the distinction between an ElectrodeTable and the ContactsTable in the probe model. The first would represent concrete physical poperties and qualities of the electrodes in the experimental rig while the second is just the contacts properties as they come from the manufacturer catalogue.

How would the schema woud look like (modified from the latest version provided by @rly, excluding the part of the AnatomicalCoordinatesTable because now we have https://github.com/bendichter/ndx-anatomical-localization).

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#ffffff', "primaryBorderColor': '#144E73', 'lineColor': '#D96F32'}}}%%


classDiagram
    direction LR

    class ExtracellularSeries {
        <<TimeSeries>>

        data : numeric
        --> unit : str = "microvolts"
        channels : DynamicTableRegion
        --> target : ChannelsTable
        channel_conversion : List[float], optional
        --> axis : int = 1
    }

    class ChannelsTable {
        <<DynamicTable (not global)>>
        --------------------------------------
        attributes
        --------------------------------------
        name : str
        description : str
        reference_mode : str, optional

        --------------------------------------
        columns
        --------------------------------------
        id : VectorData[int]
        electrode : DynamicTableRegion
        --> target : ElectrodesTable
        reference_electrode :  DynamicTableRegion, optional
        --> target : ElectrodesTable
        filter : VectorData[str], optional
        ---> Strings such as "Bandpass 0-300 Hz".
        ... Any other custom columns, e.g., ADC information
    }

    class ElectrodesTable {
        <<DynamicTable  (not global)>>
        --------------------------------------
        attributes
        --------------------------------------
        name : str
        description : str
        probe : Probe
        probe_insertion : ProbeInsertion, optional

        --------------------------------------
        columns
        --------------------------------------
        id : VectorData[int]
        contact : DynamicTableRegion
        --> target : ContactsTable
        estimated_brain_area : VectorData[str], optional
        ... Any other custom columns about physical electrode
    }

    class ProbeInsertion {
        <<Container>>
        insertion_position_ap_in_mm : float, optional
        insertion_position_ml_in_mm : float, optional
        insertion_position_dv_in_mm : float, optional
        position_reference : str, optional
        hemisphere : Literal["left", "right"], optional
        insertion_angle_pitch_in_deg : float, optional
        insertion_angle_roll_in_deg : float, optional
        insertion_angle_yaw_in_deg : float, optional
        depth_in_um : float, optional
    }


    

    namespace ProbeInterface {
        class Probe {
            }

            class ProbeModel {
            }

            class ContactsTable {
            }
    }

    ExtracellularSeries ..> ChannelsTable : links to channels
    ElectrodesTable *..> Probe : links to probe
    ChannelsTable ..> ElectrodesTable : row reference to electrode
    ElectrodesTable ..> ContactsTable : row reference to contact
    Probe *--> ProbeInsertion: might contain ProbeInsertion
    ProbeModel *--> ContactsTable
    Probe *--> ProbeModel: 

Loading

Benefits

Deduplication/Normalization of some data

Having this table allows some further normalization and avoids redundancies:

• mappings to anatomatical coordinates: paraphrasing @rly, in the neuropixel case in [Proposal] The return of ElectrodesTable #3, the channels table of ap and lp point to same electrodes and that in turns points to the AnatomicalCoordinatesTable (see here for an extended version of this argument with a concrete example) This is better than writing links from each channel to the AnatomicalCoordinatesTable and, in my own view, it aligns better with the underlying biology.

Storing physical properties of the electrode in their rightful place

Haivng this table allows to write properties in a way that make more sense conceptually.

• impedance: @bendichter mentioned the case of the impedance. Manufacturers do sometimes provide impedance measurements but they are done in specific environments that might not match the brain (such as saline solution at certain temperature). Data aquisition systems like Intan provide functionality to measure impedances in-situ so this is a good candidate for a property that should be in the electrodes table

• brain area: same, this is a property of the electrode as channels are not really physical objects. I personally think that brain_area should be the responability of the AnatomicalCoordinatesTable but maybe I am missing something?
• is_contact_active: neuropixels probes have more contacts that they multiplexing system allows to output signal from. In other words, there are more contacts that the ones recording data. Whether the electrode is active or not during an experiment is a property of the electrode in the experiment and not of the contacts table as that is determined at recording time.
• anything else?

Anything else?

Costs

More complex schema

As described in any of the many articles about normalization-denormalization trade-off there is a trade-off between deduplication and usability. I feel that deduplication arguments should have less bearing here that arguments of conceptual soundess as our data structures are neither horrible large or are subjected to a lot of concurrent writing. I would love if someone more experienced would jump in and comment along those lines, maybe @oruebel?

Name colision with classical electrodes table.

Overloading the electrodes_table name might be a bad practice.
This might be just confusing for users of the standard if electrodes_table means both the current electrodes_table that is very general and the more specific electrodes_table that we are proposing here. Maybe this is not the case as we are just rescuing the original intent usage of the electrodes table.

Other conceptual difficulties

I have some nagging doubts about the relationship with this to #7. Like, if the probe moves in the experiment, things like the impedance change. Also, the brain area of course changes so then we should have a different table? if so, conceptually it is not like we have a property of the electrode but a a property of the electrode-in-a-specic place. Something along what the ImagingPlane is doing in the ophys part of the schema.