Skip to content
Chadwick Boulay edited this page Aug 13, 2023 · 40 revisions

Socket Memory

Linux users, and some Mac users, must increase the amount of UDP socket memory available. The recommended values are 8MB default and 16 MB max. You can alternatively try to specify the buffer length via 'receive-buffer-size' but it is not recommended, and may not work or may time out.

Linux

Adjust the values (until the next reboot) as follows:

sysctl -w net.core.rmem_max = 16777216
sysctl -w net.core.rmem_default = 8388608

Adjust the values more permanently with:

echo "net.core.rmem_default = 8388608" | sudo tee -a /etc/sysctl.d/local.conf
echo "net.core.rmem_max = 16777216" | sudo tee -a /etc/sysctl.d/local.conf

Mac

  • There use to be Blackrock KB article on this, now on the way back machine.
  • kern.ipc.maxsockbuf might still be the correct parameter to change, but we might be able to get away with kern.inet.udp.recvspace. This requires more investigation. We will continue with the "old" parameter.

First try your luck and see if you can set maxsockbuf without changing boot parameter.

sudo sysctl -w kern.ipc.maxsockbuf=8388608

If the above complains that your RAM is low then try the following: (The first command here requires reboot and changes the boot parameter)

nvram boot-args="ncl=65536"
shutdown -r now
sysctl -w kern.ipc.maxsockbuf=8388608

If it still complains, or if you do not want to change boot parameters, try the following.

  • First see what the current value is:
    • sysctl kern.ipc.maxsockbuf
  • Mine is 6291456, but if yours is lower than 4194304 try this:
    • sysctl -w kern.ipc.maxsockbuf=4194304
  • If you still get errors about low RAM, try lower values.
    • If the final value is anything but 8388608, when opening the library provide it like this:
    • cbmex('open', 'receive-buffer-size', value_above)

Shared Memory

CereLink makes heavy usage of shared memory so multiple applications running on the same computer can all access the data stream coming from the device(s). If the shared memory does not exist, the first instance of CereLink will create it, and all subsequent instances will be slaves to it.

The shared memory structure is identical to that used by Central, and in fact CereLink can run on the same computer as Central because of this.

Central must be started first. Unlike CereLink, Central will always try to connect directly to the device and cannot be a slave to the shared memory from another application.

The lock file for shared memory is cbSharedDataMutex.lock, that is how multiple processes may access the same data, the first access becomes master (with UDP connection), the rest become client. This is useful only if multiple processes want to connect to the same NSP, for example a Python, an Octave, a MATLAB and a C SDK based application. Each process may have up to 4 active connections as master by providing the instance number. Lock files are crated in $TMPDIR (/tmp if not specified) and library will not run if the locks are not accessible to the user.

There are 6 shared memory files. Use ipcs -ma to show memory segments, then one can use ipcrm to remove them.

Mac: Increase the shared memory buffer

We are now using POSIX style shared memory and it seems these variables no longer require modification.

Older versions of CereLink used QSharedMemory and required the following changes:

sudo sysctl -w kern.sysv.shmmax=33554432
sudo sysctl -w kern.sysv.shmall=4194304 

These sysctl values that you found can be put in /etc/sysctl.conf (create one if it does not exist), next time OSX boots it will read the values from there and you won't need to set them again.

kern.sysv.shmmax=33554432
kern.sysv.shmall=4194304
kern.ipc.maxsockbuf=8388608

OSX Mountain Lion and Gatekeeper

  • If you are running OS X Mountain Lion tell Gatekeeper that you own the device, and are not planning to use it like a big iPod. You’ll find Gatekeeper inside the Security and Privacy pane in System Preferences, select 'anywhere' there.

Notes for Windows users:

If you wish to write an application to link to cbsdk and you are using an Express or Community edition of MSVC then you must set compiler definition NO_AFX. For example, use cmake target_compile_definitions: target_compile_definitions(my_app PRIVATE NO_AFX).

OSX distribution headaches

If the target system does not have HomeBrew installation you should make the binaries as a bundle. Bundles substitute the hard-linked dependency paths to relative paths.

First check current dependencies:

otool -L /usr/local/CereLink/cbmex.mexmaci64

And if QtCore and QtXML are not relative perfom this (Update paths to match your versions):

install_name_tool -change /usr/local/lib/QtCore.framework/Versions/4/QtCore @rpath/QtCore /usr/local/CereLink/cbmex.mexmaci64
install_name_tool -change /usr/local/lib/QtXml.framework/Versions/4/QtXml @rpath/QtXml /usr/local/CereLink/cbmex.mexmaci64 

Then verify again:

otool -L /usr/local/CereLink/cbmex.mexmaci64

To make sure inter-dependencies are also relative, first copy QtCore and QtXml to '/usr/local/CereLink' then:

otool -L /usr/local/CereLink/Qt*
install_name_tool -change /usr/local/Cellar/qt/4.8.4/lib/QtCore.framework/Versions/4/QtCore @rpath/QtCore /usr/local/CereLink/QtXml

Now you can copy the files in “/usr/local/CereLink” as a bundle.

Older version of Linux

  • We need Qt 4.6 and above because of QtConcurrentRun, older distros may not ship this version, (one may try some backport software channel, or just build newer libraries). If CCF capabilities are stripped we can use Qt4.4 as well.
  • If you use prebuilt binaries built for specific version of a distro, or if required libraries are not in your ususal distro location you can use export LD_LIBRARY_PATH=/path/to/.so/files/ before running the SDK.

To test this method do this: LD_LIBRARY_PATH=/path/to/new/qt/so/files ldd /path/to/libcbsdk.so and it should show that Qt libraries are resolved with the path you mentioned instead of system path in /usr/lib.

Remember that some applications such as MATLAB may change LD_LIBRARY_PATH, and also may be shipped with their own versions of Qt shared libraries, so you should check their version to see if that is right.

Either run the binaries as root, or do

setcap 'cap_net_bind_service=+ep' /path/to/program

In which you should give the capability to MATLAB, or Python interpreter, or your application using the sdk (neither root nor capability is not a good idea because of security reasons).

You can also use iptables to route the ports. Make sure sdk uses 51001 and 51002 bys specifying inst-port and client-port and route ports:

iptables -t nat -A PREROUTING -p udp --dport 51001 -j REDIRECT --to-port 1001
iptables -t nat -A PREROUTING -p udp --dport 51002 -j REDIRECT --to-port 1002

To delete above routing:

iptables -t nat --line-numbers -n -L

Then

iptables -t nat -D PREROUTING $num

General Notes / Trivia

Prior to 6.04 port numbers 1001 and 1002 are used. Accessing or binding to low port numbers requires root permission (since v6.04 they are changed to 51001 and 51002).