Control Message Structure

This page is to be used for design notes around the control channel.

The control channel is a separate Socket.IO channel to the data channels (The server channel sends from the server to the client, the client channel goes from the client to the server). Each instance of a uibuilder node gets it own Socket.IO session and you can limit messages to specific clients by using the _clientId (delete it if you want to send a message from a specific client to all clients connected to a node instance).

The control channel is there to keep control information separate to the general data.

The general structure of a control message is:

{ "type": "<text>" }

But other properties can be added as required.

Initially, only 2 control messages have been used. Two more have now been added. Each of these is (v0.4.8+) copied to output port 2 - you will need debug nodes to show the whole msg as msg.payload is not used. msg.topic is also added from the last input msg or from the node itself.

• type = server connected (server -> client): Sent when Socket.IO connects to client. Additional properties:

  • "debug": <boolean> - tells the client whether uibuilderfe.js should use debug mode or not
  • "_socketId": socket.id - ensures control msg only sent to the correct client

• type = shutdown (server -> client): Sent when the node instance is either redeployed or removed from Node-RED.

• type = client disconnect (server only) Additional properties:

  • "_socketId": socket.id - so we know which client disconnected
  • "reason": <string> - The disconnection reason

• type = socket error (server only) Additional properties:

  • "_socketId": socket.id - so we know which client errored
  • "error": <string> - The error

While no control messages are currently sent from the client to the server, you can do this manually by using uibuilder.sendCtrl(msg) (uibuilderfe v0.4.8+).

Note that if you reload a connected client page, you get a disconnect msg followed by a connect but they have different client ids because Socket.IO assigns a new one on connect.

What to use control messages for

Some ideas

• Send standard information to new client connections
• Keep track of utilisation of your web app (analytics, charging, ...)
• Send alerts if too many clients connect (or too few)

Questions, thoughts and ideas for future design

Questions

Please add any questions you want answering here - please start them with your initials

• JK: Should the server have a 2nd output port for passing on control messages? OR should they be passed out of the standard port. If using the standard port, should there be an option to turn them on/off?

  CDL: I suggest the 2nd output port solution. I can't see any reason to not do that, and if the user wants to use the messages then it is simpler if they appear on a different port.

  JK: Thanks Colin, I agree.

Ideas

Please add any ideas you have here for further discussion and design. Please start with your initials.

• CDL: Topic cache/repeater in the server. The suggestion is that (optionally) the uibuilder server will save the most recent payload for each topic it receives. Then on receipt of a 'reload' control message from the client it will send messages containing those topics/payloads back to the client. I think it would be useful if there were a short time delay before sending the messages back as under some circumstances the client may send multiple such requests in succession and by waiting then the requests can be combined. Maybe something like 250ms which will not be noticeable to the user. Probably there is a need for the flow to be able to tell the server to clear its cache, presumably by the flow sending a control message to the node. One further point is the the topic/payload messages sent back to the client should be identifiable as reloads rather than original in case the client app needs to know that. Perhaps add a 'uibuilder_reload' attribute to the messages.

  JK: QUESTION: Does there need to be an option to be able to cache ALL received messages? With the current suggestion, I don't think that will resolve Valter's startup issue where messages need to be retained until a client connects. This also raises some questions about sizes and how we ensure that Node-RED doesn't explode from too much data in the cache.

  CDL: Does not re-sending the latest payload for each topic solve Valter's issue? I assumed that it would, though I don't know exactly what he needs to achieve. On your question, when you ask about cacheing all messages do you mean a complete history? That would not be tenable would it?

  JK: What if a client didn't connect for some time? They might have missed several messages, possibly with completely different content. Without an option to accumulate all data, there would be a number of use cases that couldn't be solved without significant additional processing - this might be OK - I'm just asking what people think they would need. All messages could be cached with an option to send a reset message to clear the cache? Maybe would also need a memory utilisation check to force the cache clear or at least remove old messages.

  JK: FOLLOW UP QUESTION: Would it be better to keep cache processing in a separate node? I don't think most people will need it so keeping it separate would keep things simple. There are, as we've seen, also several possible modes for caching. We could also consider multiple back-ends e.g. using a db or file so that caches survive restarts. I certainly wouldn't want to do that all in one node.

  CDL: I am not sure that we aren't talking at cross purposes here. When you say "They might have missed several messages, possibly with completely different content" do you mean several messages with the same topic? If they have different topics then no, they wouldn't miss messages. For a use case such as a chart where a sequence of messages is important then the simple retransmission of the latest topic/payload values would not be enough and they would have to do something else. Let me ask a question. With the standard dashboard, when a browser connects, how do its basic widgets, text, gauge etc, get there initial state? That is the situation where I feel it would be good if it could be handled by uibuilder. I agree that more complex situations such as are needed for charts, and coping with node-red restart must be handled externally, though the restart issue is pretty much a non-issue if the user is using MQTT for state persistence.

  DCJ: Maybe (in the first instance) the replay node ought to be a separate node that responds to the "connect" control message so you don't get tied into a prescribed replay format ? And talking of control messages maybe there need to be a couple (or more)... one from the server side to say a client has connected (that then can trigger an initial download... maybe of data... but maybe prior to that - a download of initial widgets... that itself then responds with a another control message saying "and now send the initial data"