Add Kinet portout support, closes #1544 #1787
Conversation
|
@DaAwesomeP @jmusarra I think you've previously had some knowledge/use of KiNet/access to KiNet kit, so any help with testing this would be appreciated! I've only got access to one third party device which seems to have a somewhat questionable implementation. Some clarification would also be good
If we want multiple ports, I guess ideally we want multiple devices too (just so the OLA ports match the KiNet ports really): |
|
I have some CK devices at home and at work; I'll take a look next week! |
|
As sort of mentioned, currently its just some WIP code (I didn't think anyone would reply so quickly). Changing Line 46 in fcff42a But more generally as someone who has or has used the real kit, input on some of the more general operational queries above would be appreciated. Having not really had hands on with proper kit I don't know if we need to actually implement universes for example. |
…to kinet-port-out
…nto kinet-port-out
|
Disclaimer: I am not super knowledgeable about the technical details of the protocol, but I can help with the functions and try things out. Introducing some Color Kinetics-specific vocabulary to avoid confusion:
So with two strings of 50 fixtures each plugged into my PDS-60ca, light 1 on port 1 is address 1. Light 1 on port 2 is address 151. So for this particular case you would not need specific port packets or multiple universes. However, if you have more channels on the strings than fit in one universe then you would need another method; in that case I think you have to use the port out commands. I think in most situations you would send to each port of a power supply separately because you usually have more than a few lights on a port. There are definitely cases were you only use one port on a power supply (if you have only one string of fixtures or they fit in one universe of non-port out commands), but if you are using both ports it is very likely that you want separate data to them with separate port out commands.
On the PDS-60ca I have in front of me at the moment, I can set a universe for the whole device but it applies to both ports. This means that in order to control each port separately, you need to use the port out commands or have few enough lights to fit on one universe. The OLA that currently ships with Debian 11 writes to the single universe no matter which universe number is assigned in OLA or on the PDS. The universes I think come in handy for fixtures that are combination KiNet+fixture all-in-one, like a ColorBlaze TRX. In this configuration KiNet functions not that different from an individual sACN fixture where each fixture has a universe and a starting address and I am pretty sure packets are sent multicast. I should note that the TRX is more of a live lighting/live event light and not a device targeted at installation like the PDS-60ca.
So there seem to be a few different methods of sending data:
So methods 1 & 2 are more traditional universe and 3 & 4 are by port. I believe methods 1 & 2 are mutually exclusive from 3 & 4, but I don't think this needs to be represented as mutually exclusive in the UI. I'm thinking that instead one should show up as "KiNet universes" and one as "KiNet Port Out." So by default simply "enabling KiNet" causes options similar to sACN to show up. If you add in a specific IP address, then it would additionally show methods 1 and 3 (and possibly 4). For method 3, it should show each port separately. I think you can encompass method 4 by simply checking both method 3 ports into one universe in OLA.
Definitely. This is what Quick Play Pro does. It detects the IPs of devices. It can also initiate a fixture scan on the PDS where the PDS figures out how many lights are plugged into it, but that is less necessary. The most important parts are finding the IPs and how many ports are available on each IP.
I can only see this being necessary if you are interfacing with a CK or other brand of playback device. I suppose it would be very useful for debugging but probably not as much as a missing feature as the above parts. You would probably only want to implement methods 1 & 2 as input or you would have to add some strange port logic I suppose to OLA.
I am personally happy to just fire data at it as if it was sACN gateway (I suppose without RDM). You definitely need to know how many ports the PDS has and be able to assign them to separate universes, but I don't see a use for scanning the fixtures on the PDS or other configuration parameters. I feel like that will be a constant back-and-forth of reverse engineering as new devices come to market. As far as OLA appearing in Quick Ply Pro, personally I can't see a need for it. Quick Play has a dropdown for specific devices from CK, so I could see that breaking with OLA pretty easily. I think it would be strange to go to the trouble to install a whole system and use a CK playback unit and need it to talk to OLA. |
|
TLDR:
Method 2 is most necessary for devices that are effectively self-contained fixtures where multiple KiNet devices sit on one universe (like sACN universes). Method 3 is most necessary for install systems with long chains of fixtures plugged into them. For method 3, each port should be able to be assigned to a universe in OLA (and preferably multiple ports on one universe). |
|
I apologize for the message spam, but I hope this is all immensely helpful since I have the devices hands-on. Talking to some acquaintances who also work with these devices, the KiNet spec is under NDA (currently shared with other close-source providers that implement it), but they also said may not actually hurt to ask CK about open sourcing at least part of it given how much progress has been made in reverse-engineering the V2 spec. This pull request is a good demonstration of that progress. Personally I have not seen the spec. I have also confirmed that the universe method is v1 and called "DMXOUT" and the v2 method is "PORTOUT." The PDS-60ca supports both KiNet v1 (universes) and v2 (ports) so I can effectively test everything with it. The PDS-480ca, which I may use in a few months, only supports v2, so I am very interested in that. I have confirmed that the DMXOUT mode on the PDS-60ca performs as I tested it: lights from both ports are placed back-to-back on one universe. But apparently this is deprecated and PORTOUT is the way to go. I'm thinking that treating KiNet v1 and v2 entirely separate (separate config files in OLA) is the way to go. That way you can mix the devices if you need to. The two protocols also work very differently (universes vs devices w/ ports). |
|
That's incredibly useful already thanks @DaAwesomeP as you say knowing the terminology and the explanation from using it in practise is something I've not had the benefit of. I'll reply in more depth to all your comments soon. So in terms of testing, this PR should currently just implement options 3 (and 4 as you say by patching them all to the same OLA universe). Currently it generates 16 ports (from 1-16) for each PSU IP (with those configured in the same way as we used to for DMX out PSUs. I'd appreciate it if you get a chance to test that (I've checked my data against packets received from my test, non-CK, fixture, but I've not actually tried sending data back to it yet). In theory it should work though... 🤞 If so then I can work on finenessing all the other options too... |
|
Great! I'll try building it tomorrow. |
As mentioned, still super useful.
So if you sent port out, would you expect that offset to apply relative to the two ports (i.e. it does it itself)? Can you have more than (512/3) pixels on one physical port, if so what happens there, or can the PSUs not support that number of lights?
Ah okay, so OLA is already using some sort of unicast send to the PSU, but sending to the "any universe" universe on the device, so it always receives the data on the PSU regardless of which universe that's set to. OLA certainly currently ignores the OLA universe when sending KiNet data. What range of universes can you set on the device? Does it start at 0 or 1?
A packet capture of that data would be invaluable, to find out where it multicasts to. Does it always multicast to one IP, and just change the universe data in the packet, or is there an IP per universe to multicast to too?
Yes, that OLA send is currently unicast to the "broadcast universe" at that IP address.
Okay, aside from pinning down some of the details above that would be a fairly easy change to add.
Yeah that's what this new PR currently implements.
I assume some PSUs could have even more than just two ports?
I think logically config wise, 2 should be separate from 1 and 3 (and 4), I was more leaning towards with 1 and 3 you have to set an IP, and when you've set one you can configure to send to that IP by DMX or by port out; currently if you picked portout it would generate a device with 16 OLA ports, where each corresponds to a different KiNet port on that device.
Agreed, that will definitely work.
Ah I didn't realise you could find the port count too. Do you have two different devices with different port counts you could test with? We'll also need to allow manual config too, if anyone wants to send to an unofficial device or one which doesn't respond.
Okay, definitely a lower priority then, and easier to do at least some testing with Quickplay Pro of my own accord. Well we could make an OLA dummy CK device which just has 16 ports say, and they'd appear as OLA ports so you could patch those if you needed to receive port out data.
How does Quick Play Pro work with other manufacturer then? |
|
OK, I built the new version, but I'm not seeing the new ports:
enabled = true
power_supply = 192.168.1.250I guess maybe I misunderstood your code note above, do I need to change that line and then this listed KiNet device will output to that fixed port? |
Since this is a device with two physical ports but I am connecting in DMXOUT mode, it places the fixtures from both ports in one universe. In PORTOUT mode, each physical port would be completely separate with the devices starting at address 1.
I would assume that no, you cannot have more than 512 addresses on a port. The datasheet for the largest PDS that I know of, the PDS-480ca, says that it can support no more than 75 fixtures on a port (225 addresses for 3-channel fixtures): https://www.docs.colorkinetics.com/support/datasheets/sPDS-480ca_ProductGuide.pdf
That matches my current experience with OLA.
The textbox in QuickPlay Pro allows me to set from 0 to 65535 inclusive on the PDS-60ca.
I think it is broadcast (to the subnet broadcast address) and it changes the packet contents. I am not sure if QuickPlay Pro will produce this packet or not, but I can try. It appears that in the left-hand pane of the "Test" page that I can select all Ethernet devices at once and send data to them. It seems to being using the DMXOUT method to do this since I cannot select a port, but it also doesn't let me select a universe (so it is maybe either using PORTOUT to all devices or DMXOUT to all universes). The old version of QuickPlay Pro might be able to do this. I have some older DMXOUT-only devices at home that I can try playing around with. I will think about what devices I may have that output a broadcast packet. I think this is the only part of the DMXOUT part of OLA that is missing.
Definitely. The PDS-480ca has 8 ports!
I see. This prevents you from simultaneously sending a unicast DMXOUT and unicast PORTOUT to the same device as only one would appear in the interface. Maybe it could be something like (similar to OSC): enable_broadcast = true
power_supplies = 2
power_supply_1_ip = ip
power_supply_1_mode = portout
power_supply_1_ports = 8
power_supply_2_ip = ip
power_supply_2_mode = dmxout
I'm not certain if QuickPlay Pro is doing a lookup of the device type or if the device says how many ports it has. QuickPlay Pro definitely can identify models from serial numbers, as you can add virtual devices by serial number or model. I do not have two different types of PORTOUT devices at the moment, but I will most likely in a few months.
Sounds good! I think the DMXOUT input would also probably be simple to implement and similarly lower priority.
Ah, I didn't think about that. I do not have any non-CK devices. However, in QuickPlay Pro if you click add Ethernet device, it does not show a "generic" or other method of adding another manufacturer's device (or maybe some of the ones listed are from other manufacturers). You can add a "Generic 3-ch" or etc. fixture to an Ethernet device though. How does it work with your device? This also might be worth checking in the old version of QuickPlay Pro. |
Ah, ok I seem to have been using the existing version of |
Did you run it as Can you share olad -l 4 logs which will confirm that? |
|
OK, I removed the OS-installed OLA and ran it with: $ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/myuser/GitHub/ola/olad/.libs ./olad -c ~/ola -d /home/myuser/GitHub/ola/olad/www -p 9091
/home/myuser/GitHub/ola/olad/.libs/olad: error while loading shared libraries: /home/myuser/GitHub/ola/olad/.libs/libolaserver.so.0: file too shortIt seems that file is indeed smaller than it should be: $ ls -al .libs/
total 97836
drwx------ 2 myuser myuser 4096 Sep 14 13:37 .
drwxr-xr-x 7 myuser myuser 4096 Sep 14 13:37 ..
-rw------- 1 myuser myuser 49428096 Sep 14 13:37 libolaserver.a
-rw------- 1 myuser myuser 18 Sep 14 13:37 libolaserver.la
-rw------- 1 myuser myuser 1273776 Sep 14 11:58 libolaserver_la-AvahiDiscoveryAgent.o
-rw------- 1 myuser myuser 435176 Sep 14 11:58 libolaserver_la-ClientBroker.o
-rw------- 1 myuser myuser 158216 Sep 14 11:58 libolaserver_la-DiscoveryAgent.o
-rw------- 1 myuser myuser 224048 Sep 14 11:58 libolaserver_la-DynamicPluginLoader.o
-rw------- 1 myuser myuser 293264 Sep 14 11:58 libolaserver_la-HttpServerActions.o
-rw------- 1 myuser myuser 1384 Sep 14 13:37 libolaserver.lai
-rw------- 1 myuser myuser 2013288 Sep 14 11:58 libolaserver_la-OlaDaemon.o
-rw------- 1 myuser myuser 6857328 Sep 14 13:36 libolaserver_la-OladHTTPServer.o
-rw------- 1 myuser myuser 3211256 Sep 14 11:58 libolaserver_la-OlaServer.o
-rw------- 1 myuser myuser 3197904 Sep 14 11:58 libolaserver_la-OlaServerServiceImpl.o
-rw------- 1 myuser myuser 733352 Sep 14 11:58 libolaserver_la-PluginManager.o
-rw------- 1 myuser myuser 11969600 Sep 14 11:59 libolaserver_la-RDMHTTPModule.o
-rw------- 1 myuser myuser 21 Sep 14 13:36 libolaserver.so
-rw------- 1 myuser myuser 21 Sep 14 13:36 libolaserver.so.0
-rw------- 1 myuser myuser 19436720 Sep 14 13:36 libolaserver.so.0.0.0
-rwx--x--x 1 myuser myuser 819232 Sep 14 13:37 oladI configured with: $ ./configure --disable-all-plugins --enable-e131 --enable-kinet --enable-dummy --enable-osc --enable-artnet --enable-shared --enable-staticAnd build with: $ make all -j4 |
|
My apologies...this is some KVM filesystem shenanigans. I have it all listed now: |
|
OK, so unfortunately it does not seem to work. Nothing happens when I manipulate the sliders for both universe/port 1 and 2.
When running with |
|
If we are using C++17 then the fix is:
class KiNetNode {
private:
static constexpr uint16_t KINET_PORTOUT_MIN_BUFFER_SIZE = 24;
}; |
That doesn't sound great, would you mind making a new issue to track it, particularly because as you say it might impact other OLA plugins.
Spot on, removing that for this usage in this build here clears the problem: So at least we know what needs resolving now! |
|
Aha! Great! I think we should leave in the I'm sure something like clang-tidy or other more strict static analysis would find this sort of C++ non-compliance bug in the future if we want to go that route. However the number of changes across the repo would be drastic unless it was done in small parts. |
|
Opened peternewman#44 to resolve the variable issue. |
KiNET Fix KiNetNode Static Private Constant Variables
|
Lint fails as expected from a codespell typo not from this pull, but the Debian build now passes! |
|
@peternewman polite ping! |
Thanks very much for the other PR. I wasn't proposing removing the
Sorry for the slow action, I've been away and also wanted to clean up a few outstanding todos. I think this is now good to merge if @kripton gets a chance to take a look too... |
|
@peternewman ready to merge? |
I'd like the new stack of tests to run first, which will need a resync PR from 0.10 to master, which I can do in due course unless you're keen enough you want to generate one (in your own branch) for now? |
Makes sense! I'll take a try at merge 0.10 into master, or at least my |
|
@peternewman OK, that wasn't so hard: #1860 |
|
Amazing! Build is passing! |
|
Thanks for doggedly working to get this in @DaAwesomeP , now onto the next challenge...! |
No description provided.