trial_continuous does not seem to return consecutive chunks of data

[stinos]

Test code (ran against v7.0.4 on a 256ch Cerebus with no spike extraction and no raw data streams and continuous acquisition enabled only for elec1 at 1kHz and ainp2 at 30kHz):

from cerebus import cbpy

cbpy.open()
sysConfig = cbpy.get_sys_config()
ticksToMSec = sysConfig['sysfreq'] / 1000.0

# Convert cbpy tick to mSec.
def ConvertTime(tm):
  return tm / ticksToMSec

def CbpyTime():
  return ConvertTime(cbpy.time()[1])

bufferParameters = {'absolute': True, 'event_length': 16384, 'continuous_length': 102400}
cbpy.trial_config(reset=True, nocomment=True, buffer_parameter=bufferParameters)

data = []
for _ in range(10):
  # Gather chunks of about 50mSec worth of data.
  startTime = CbpyTime()
  while CbpyTime() < startTime + 50.0:
    pass
  data.append(cbpy.trial_continuous(reset=True))

cbpy.close()

for _, channels, timestamp in data:
  timestamp = ConvertTime(timestamp)
  for channelId, channelData, samplingRate in channels:
    numSamples = len(channelData)
    print(f"ch {channelId:03d} numSamples {numSamples:04d} start {timestamp:0.1f} end {timestamp + numSamples * 1000.0 / samplingRate:.01f}")

Sample output:

ch 001 numSamples 0041 start 104031352.0 end 104031393.0
ch 258 numSamples 1230 start 104031352.0 end 104031393.0
ch 001 numSamples 0059 start 104031403.0 end 104031462.0
ch 258 numSamples 1770 start 104031403.0 end 104031462.0
ch 001 numSamples 0051 start 104031457.0 end 104031508.0
ch 258 numSamples 1530 start 104031457.0 end 104031508.0
ch 001 numSamples 0059 start 104031513.0 end 104031572.0
ch 258 numSamples 1770 start 104031513.0 end 104031572.0
ch 001 numSamples 0050 start 104031565.0 end 104031615.0
ch 258 numSamples 1500 start 104031565.0 end 104031615.0
ch 001 numSamples 0050 start 104031622.0 end 104031672.0
ch 258 numSamples 1500 start 104031622.0 end 104031672.0
ch 001 numSamples 0050 start 104031672.0 end 104031722.0
ch 258 numSamples 1500 start 104031672.0 end 104031722.0
ch 001 numSamples 0050 start 104031722.0 end 104031772.0
ch 258 numSamples 1500 start 104031722.0 end 104031772.0
ch 001 numSamples 0051 start 104031772.0 end 104031823.0
ch 258 numSamples 1509 start 104031772.0 end 104031822.3
ch 001 numSamples 0049 start 104031822.0 end 104031871.0
ch 258 numSamples 1491 start 104031822.0 end 104031871.7

Am I interpreting this correctly?

• there is a gap of 10mSec between the end of the first data chunk and the beginning of the next one
• for the last 2 chunks a different amount of data is returned for the 2 channels, and based on the timestamps there might be duplicate samples in there (because end of one chunk is 104031823.0 but the next one starts at 104031822.0 again)

Is there a way to reconstruct a continuous data stream from trial_continuous? I know I'm on a fairly old version, did anything change with respect to this particular functionality in more recent versions?

[cboulay]

A few comments:

1. Please use the latest version of the code but configure the CMake project with -DCBPROTO_311=ON. This gives you the latest version of the code that's backwards compatible with devices running firmware 7.0.x. I won't be supporting old versions of the code. If you encounter the same issue there then I can investigate further.

2. You may also be interested in trying pycbsdk. It is also compatible with both 3.11 and 4.1. With that library I'm sure it isn't losing data. However, there is a big caveat that the pycbsdk-running process can be the only process on that machine that is pulling data from the NSP. No Central, no CereLink, and no other pycbsdk process will work at the same time on that machine (other PC on the network running Central is fine).

3. The reset=True always confuses me. What happens if you do reset=False in the loop?

[stinos]

1. Ok will try, but will only come back to this in a couple of weeks
2. Interesting but not sure how feasible it is for us to convert all existing code
3. Not 100% sure but I think reset=True means after getting the samples, discard them whereas reset=False would return everything since the last reset i.e. just everything if never reset, up to continuous_length

[stinos]

Please use the latest version of the code but configure the -DCBPROTO_311=ON

Using current master had to fix the cmake line which adds compat to the include directories:

target_include_directories(${LIB_NAME} PUBLIC $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/compat>)

should be

target_include_directories(${LIB_NAME_STATIC} PUBLIC $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/compat>)

Then built with

conda create -n py39 python=3.9 numpy Cython pip wheel setuptools
conda activate py39
set QTDIR=c:\tools\qt\6.5.2\msvc2019_64
"c:\Program Files\CMake\bin\cmake.exe" -B build -S . -G "Visual Studio 17 2022" -A x64 -DCMAKE_INSTALL_PREFIX=./install -DCPACK_PACKAGE_DIRECTORY=./dist -DCBPROTO_311=ON -DBUILD_STATIC=ON -DBUILD_CBMEX=OFF -DBUILD_CBOCT=OFF -DBUILD_TEST=OFF -DBUILD_HDF5=OFF -DCMAKE_PREFIX_PATH=%QTDIR%
"c:\Program Files\CMake\bin\cmake.exe" --build build --config Release -j4
"c:\Program Files\CMake\bin\cmake.exe" --build build --target=install --config Release
copy install\bin\Qt*.dll cerebus\
pip -vv wheel . -w dist/

The target PC has the exact same conda environment, the whl install fines, cbpy.open works but calling cbpy.trial_config results in a crash (0xc0000005 i.e. access violation), also with its default parameters. Maybe has to do with CBPROTO_311, has this been fully tested against older hardware? I'm running with Central v7.03 and NSP Firmware v7.0.5.

edit also tried the latest code plus some changes to make it build with Qt5 and Python 3.7 and the same numpy version as the original Python environment which I used for the sample code in the first post, to try to rule out any issues there, but that has the same problem