The protocol for the interaction between the ctGateway gateway and the platform

1 Overview

1.1 General Description

This document is an interaction protocol between the gateway and the platform, which is first implemented on the gateway and the gateway management tool.

The protocol MQTT is based on connectionless MQTT messages, and the messages are UTF-8 encoded JSON data structures (involving different encodings that are specially processed in packets).

The protocol is planned to be extended to HTTP and the Universal Serial Bus, still maintaining the same message processing pattern.

The protocol includes device management and data reporting.

Device management includes gateway registration, gateway update, gateway configuration, and status query.

Data operations include real-time reporting, historical reporting, data confirmation, and data writing.

1.2 Basic Data Format

Basic format:

Data items	platform	function	Required	illustrate
func		function	be
ts		timestamp		The timestamp of the packet generation
uid		Unique Identification		Uniquely identify packets
data		Data items		Unique data for each func

A data series is a sequence of data reported by the gateway, which increases monotonically. When the gateway registers, it reports a starting data series value of -1 (which corresponds to the maximum value of data that has already been acknowledged, and if a data drop occurs, this value indicates that the data no longer exists). It has been confirmed that the data cannot be reported again.

The platform sends data confirmation messages to the gateway at regular or quantitative intervals. If the platform does not confirm the data for a long time, the gateway may drop some of the data or stop the collection.

The data series is incremented from 1.

1.4 Compression and Encryption

Common data is in JSON format. Compressed data starts with a capital letter "Z", encrypted data starts with an uppercase letter "E", and data that is compressed first and then encrypted starts with a capital letter "EZ".

The gateway does not compress by default, and does not encrypt reg, hb, and push by default.

The gateway uses the agreed password to encrypt other reported data. The gateway has an initial password and can be changed through the platform command.

2 Agreement

2.1 General Classes

2.1.1 Respond

It is used for the gateway to respond to various instructions issued by the platform, and some instructions provide for special responses, which should be supported by both general and dedicated response platforms.

Answer Packet Format:

Data item	function	Required	illustrate
func	function	be	The func corresponding to the command issued
uid	Unique Identification	be	The UID of the corresponding command
state	state	be	0: Failed 1: Succeeded
error_code	Error codes		If it fails, we recommend that you carry an error code
message	Additional Information		It may be an error message, or it may be the result of an execution

{

"func": "respond",

"ts": 1706676012126,

"uid": "1e0b8d3d86f28702bda02f6998897bdb",

"data": {

"func": "sudo",

"UID": "UID of the request packet".

"state": 1,

"error_code": 0,

"message": ""

}

2.1.2 Loglevel

Specify the log level for the gateway.

Data item	function	Required	illustrate
loglevel	Log level	be	0 is normal and 1 is debugging. Valid only for running programs, and will expire after restart.

For debugging, the gateway does not need to answer.

2.1.3 Set the timeout overtime

Set the processing time of the instructions issued by the platform (some instructions may cause deadlocks and affect the operation of the gateway, if the processing cannot be completed due to the processing time limit, the timeout setting can be increased appropriately).

Data item	function	Required	illustrate
overtime	Timeout Period (seconds)	be	Greater than or equal to 30

2.1.4 Report

It is used for the escalation of information that has not yet been formalized.

Data item	function	Required	illustrate
interface_code	interface		The relevant interface, if not filled, belongs to the overall information
device_code	equipment		If it is not filled, it belongs to the entire interface
channel_code	passage		If it is not filled, it belongs to the entire device
messsage	message	be	Text of the message

2.2 Gateway Management Class

2.2.1 Gateway Discovery

When the gateway is powered on, the registration information is also sent to the dedicated topic: device_gateway/reg, which is only used for gateway discovery.

The registration information is the same as the reg.

2.2.2 The gateway registers the reg

Initiated by the gateway, it registers itself with the platform and reports its serial number, hardware version, and software version.

The platform returns whether the registration was successful or failed.

Gateway Request Packet Format:

Data item	function	Required	illustrate
sn	Serial number	be	Gateway serial number IMEI
model	Model		Hardware model, 5G gateway, 4G gateway, wifi gateway
sw_version	Software Version	be	The version of the gateway program
iccid	SIM card number		SIM card number (if any)
latitude	latitude
longitude	longitude
key_seq	The configured key identity
current_key_seq	The actual key identification
data_sequence	Data series		A confirmed sequence of data that has been discarded on the gateway and cannot be made available again
	uid
start_time	The time when the program started		The start time of the program
uptime	System startup time		The boot time of the operating system

Platform Answer Packet Format:

Data item	function	Required	illustrate
code	Processing status	be	0 successes, 1 failures

2.2.3 Notregs are not registered

Initiated by the platform, it is delivered when the platform thinks that the gateway needs to be registered. The gateway may have to register before it can send additional packets (as determined by the platform), and the platform may close the gateway connection immediately after sending this packet, or it may maintain the connection and discard all non-registered packets. The gateway receives this packet and should immediately re-register or reconnect to the platform for registration.

Package Format:

Data item	function	Required	illustrate
			There is no data item

{

"func": "notreg",

"sn": "ABCD1234567890"

}

2.2.4 Report the state

The platform now proactively escalates for the gateway. 2 - Polling 3 - No Configuration 4 - Motherboard Communication Abnormal 5 - Point Read Abnormal

Changed to a platform request.

Platform Packet Format:

Data item	function	Required	illustrate
			There is no data item

{

"func": "state"

}

Gateway Packet Format:

Data item	function	Required	illustrate
gw_state	Gateway status
mobile_type	Connection type		5G/4G/Wifi
csq	Signal strength
interface_state[]	Device status		The status of each interface

Interface_state format:

Data items	function	Required	illustrate
interface_state	Interface status		Connected, Not Connected, Connection Failed
interface_code	The name of the interface
device_state[]	Device status		The status of each device, including the device with the error

Device_state format:

Data items	function	Required	illustrate
device_state	Device status
device_code	Device identification	be
channel_state[]	Device status		The state of each channel, including the channel with errors

Channel_state format:

Data items	function	Required	illustrate
channel_state	Channel status
Channel_write_state	Channel write status
channel_code	Channel identification	be

{

"func": "state",

"ts": 1706681745689,

"uid": "529a9d42abf8bc75b1fde02ec59e3d01",

"data": {

"gw_state": 0,

"mobile_type": "5G",

"csq": "27",

"interface_state": [

{

"interface_code": "",

"interface_state": 1,

"device_state": [

{

"device_code": "device503",

"device_state": 0,

"channel_state": [

{

"channel_no": 0,

"channel_code": "code1",

"channel_name": "Specification",

"channel_state": 1,

"channel_write_state": 0

}

]

}

]

{

"interface_code": "",

"interface_state": 2,

"device_state": [

{

"device_code": "La1",

"device_state": 0,

"channel_state": [

{

"channel_no": 0,

"channel_code": "0",

"channel_name": "",

"channel_state": 0,

"channel_write_state": 0

{

"channel_no": 1,

"channel_code": "1",

"channel_name": "testBit",

"channel_state": 0,

"channel_write_state": 0

}

]

}

]

}

]

}

2.2.5 Report heartbeat HB

The gateway takes the initiative to report. It is only sent when no information has been reported for a period of time.

No special data is required.

{

"func": "hb",

"ts": 1706681046297,

"uid": "58261d73bcbf864005185c4f814c9aa0",

"data": {

"hb": 1,

"ts": 1706681046298,

}

2.2.6 Report the fault report

The gateway reports an exception.

Data item	function	Required	illustrate
interface_code	interface	not	The network port or serial port ID associated with the message
device_code	equipment	not	The device ID to which the message relates
channel_code	passage	not	The channel ID associated with the message
message	message	be	The content of the message

2.2.7 Firmware upgrade FOTA (not implemented)

Upgrade the gateway software, initiated by the platform, and provide a download link. The gateway replies that the upgrade succeeded or failed.

2.2.8 Reset the protocol reset

The protocol configuration of the gateway is cleared, initiated by the platform, and the gateway answers.

{"func":"reset","data":{"seq":"6a67acf6935040c4bf78384c9ce2d628"}}

The gateway program takes effect the next time it starts.

2.2.9 Specify the conf configuration

Data structure:

Data item	Child	function	Required	illustrate
protocolCode		Protocol identifier	be
interfaceCode		The name of the interface	not	Serial port name, non-serial port protocol can be omitted
resolvePropertyMap		Interface parameters	be	The channel ID associated with the message
	rate	Acquisition frequency	be	second
(non-serial)	ip		be
(non-serial)	port		be
(Serial)	serial_port	Serial name		It cannot be left blank at the same time as the interfaceCode
	baudRate	baud rate	be	Up to 115200
	parity	verify	be	N E O
	dataBit	Data bits	be	5 6 7 8
	stopBit	Stop bits	be	1 2
	timeout	Response time	be	Actually unused (library problematic)
confTabs[]		Array of devices	be	See the table below

confTabs:

confTabs[] item	Child	function	Required	illustrate
deviceCode		Device identification	be	Globally unique
encodeType		Encoding type		Used only for tunneling protocols, HEX ASCII
stationCode		Slave station number		For non-tunneling protocols only
resolveParamConfigVOList[]		Array of channel lists	be	second
	paramCode	Channel identification	be
	paramName	The name of the channel	be
	paramCommand	directives		Used only for the tunneling protocol, and the tunneling protocol does not use all subsequent fields
	dataType	data type	be	From this field below, only non-tunneling protocols are applied
	paramAddr	address	be	It is based on 0 with functionCode and 1 with zone (only for configuring from csv).
	functionCode	Feature codes	be	Pay attention to the relationship with the PCL partition
	dataLength	The length of the data	be	Actually only for the string type
	processType	Read and write type	be
	reportStrategy	Escalation type	be	1-Report by collection frequency 2-Change report 3-Listen report (not yet supported) 4-Conditional report (not yet supported) 5-Dependent report 6-Change trigger full volume
	transferRuleStr	Endianness		ABCD AB works with all data types, including string
	associationParamCode	The identity of the dependent channel		Used for dependency escalation

Features that have not yet been normalized:

item	platform	function	Required	illustrate
zone		PLC partition		Currently only used in CSV, the address is based on 1 when configured with this field (the directive is automatically corrected when executed by the gateway)
collectorRate		Percentage of change reported in the frequency of acquisition or change		Currently only used in CSV, changes above this value are reported
charsetCode		String encoding		At present, it is only used in CSV, the encoding used by the device, the interaction between the gateway and the platform is always UTF-8, but the encoding used by the device may be a local encoding, if this value is set, the gateway will try to do the encoding conversion

2.2.10 Delivering Network Configuration IP (Not Fully Implemented)

The platform delivers the issue, and the gateway responds.

Delivery Format:

{"func":"ip","data":{"phyConnect":"a","ip":"b","gateway":"c","seq":"1706682377329","ts":1706682377329}}

Due to hardware differences, the serial port name and network card name are uncertain, and there is currently a lack of communication mechanism with the platform, so the gateway only processes IP for the time being. IP is dotted format +"/24".

Note that since the gateway is modified by executing shell commands, a successful response by the gateway does not necessarily mean that the operation is successful.

The gateway supports additional options:

Data item

function

Required

illustrate

operate

not

add (default.))

del delete

show lists existing IPs

2.2.11 Sending Wifi Configuration to WiFi (Not Implemented)

The platform delivers the issue, and the gateway responds.

2.2.12 Deliver the configuration connt for the northbound platform connection

The gateway program takes effect the next time it starts. Since the gateway needs to support different platforms, some protocol modifications need to be made.

Data item	function	Required	illustrate
comment	illustrate
type	Connection type	be	MQTT MQTTS
platform	Platform code		MQTTS is required, and this value affects the certificate used
hostname	host name	be
port	port	be
userid	Username	be
upwd	password	be

2.2.13 Restart the gateway

The platform initiates and restarts the gateway. Note that if this command is executed successfully, you may not receive any reply (because the gateway has already been restarted).

No additional data is required.

2.2.14 Restart the program reload

Restart the program and reload the configuration.

No additional data is required.

2.2.15 List (Hidden Function)

An actionable list of gateways, currently a list of files that the gateway is available for download.

Platform delivery format (no data is required):

Data item	function	Required	illustrate
dir	directory	not	The directory name, absolute or relative, defaults to the working path of the gateway

The gateway reports a UID containing the request for identification.

Data item	function	Required	illustrate
file_list[]	filename	be	List of files

file_list Structure:

Data items	function	Required	illustrate
file_name	filename	be
is_dir	Whether it's a directory or not	be	1: Table of Contents
file_size	File length	be
file_time	File time	be	Last modified time YYYY-MM-DD HH:MM:SS
file_checksum	File checksums	be	MD5-16 bytes, HEX strings

2.2.16 Download file download (hidden function)

Download the file for the gateway.

Platform Request Format:

Data item	function	Required	illustrate
file_name	filename	be	filename
dir	Directory path		Relative or absolute
private_data	Loopback field		The original loopback in the reply is actually the corresponding local path

The gateway answers:

Data item	function	Required	illustrate
file_name	filename	be
file_size	File length	be
file_content	File contents	be	Base64 encoding
file_time	The time when the file was modified	be	YYYY-MM-DD HH:MM:SS
private_data	Loopback field		If it exists in the request, it is returned as-is
content_start	Start location		The content start location, if it's full content, it doesn't need to be sent
content_length_	Content length		Must be sent with content_start

2.2.17 Upload File (Hidden Function)

Upload the file. Additional certifications may be required.

Data item	function	Required	illustrate
file_name	filename	be
dir	directory		Relative or absolute paths
file_size	File length	be
file_content	File contents	be	Base64 encoding
file_time	The time when the file was modified	be	YYYY-MM-DD HH:MM:SS
content_start	Start location		The content start location, if it's full content, it doesn't need to be sent
content_length_	Content length		Must be sent with content_start
file_checksum	File checksums		MD5-16 bytes, HEX strings Sent with the last segment

The gateway replies with a generic answer. The last reply will tell you whether all the processing has been successful. If the file validation fails, the file is not actually written (or if the segments arrive out of order).

2.2.18 Sudo (Hidden Feature)

Execute console commands.

2.2.19 Show (Hidden Feature)

Displays internal data.

2.2.20 Enable enablecompress

Compression is enabled (small packets are longer after compression, so they are selected according to the actual effect), the first character of the compressed packets is 'Z', followed by base64 encoded data, and the data is compressed by zlib:

Data items
Z (uppercase)		Leading, uncompressed packets always start with curly braces
Base64 data		The following data is encoded by base64
	Length after unzipping	8 bytes, the endianness of the gateway
	Compress data	Blocks of data compressed using zlib (not zip files)

No additional data is required to issue this command.

2.2.21 Enable enablecrypt

With encryption enabled, all data is encrypted in transit (if it has been compressed, the encrypted data is compressed). The encryption algorithm is AES256-CBC. The default control directives are already encrypted.

Data items
E (uppercase)		Leader
Base64 data		The following data is encoded by base64
	Plaintext length	8 bytes, the endianness of the gateway
	Padding bytes	8 bytes
	Initial IV	16 bytes
	Encrypt data	Multiples of 16 bytes, with the last block deficiency filled with 0

No additional data is required to issue this command.

2.2.22 Update Key (Confidential)

A new key is issued. The gateway has been implemented, and it will not be disclosed for the time being because the supporting facilities have not yet been completed.

2.3 Data Manipulation Classes

2.3.1 Data reporting push

Follow the platform interface.

Supplemental data:

Data array items	meaning	Required	illustrate
type	Escalation type	be	1 is full collection, including all readable channels, and 0 is others
events	Trigger an event		The channel code where the event was triggered, separated by spaces: The dependent channel of the associated escalation Change the channel in which all are uploaded Note that there is no change during full collection, and there will be no trigger event
ms	Take		The time taken for this processing is measured in milliseconds
count	Number	be	The number of reported data

2.3.2 Reacquisition acquire

No additional data is required. The gateway immediately initiates an all-time acquisition.

2.3.3 Data confirmation

The platform delivers the received data serial number, and the gateway can discard the serial number and all previous data after receiving it.

Data item	function	Required	illustrate
data_sequence	Data series		The largest series of data that the platform has received

The NMS uses a generic reply to reply to the processing status.

2.3.4 Request to re-escalate missing

The platform delivers the missing data sequence, and the gateway resends it. If the specified data sequence does not exist, the gateway will re-register. If the data exists, the gateway pushes again.

Packet Format:

Data item

function

Required

illustrate

data_sequence

Data series

Comma separates, and dashes indicate ranges

Example: "2,6-10"

2.3.5 Write the platform write point

The platform requests, and the gateway returns the execution result.

Data is an array, and the properties of each element are as follows:

Data item	function	Required	illustrate
deviceCode	Device identification	be
paramCode	Point identification	Choose one of the two
channelNo	Channel number	Choose one of the two
value	The value to be written	be

2.3.6 Tunnel Protocol Write Instruction send

Can only be used with tunneling protocols.

Data item	function	Required	illustrate
deviceCode	Device identification	be
dataValue	Instructions to write	be	The encoding must be consistent with the device attributes (HEX or ASCII)

2.4 Specific Protocol Classes

2.4.1 Connecting is being connected

Used for tunneling protocols to report the connection status.

2.4.2 Connection failure connecting_failed

Used for tunneling protocols. If the gateway automatically reconnects after the connection fails, you can send a reload command to restart the program (equivalent to a fast reconnection).

Due to traffic issues, this message is no longer sent.

2.4.3 The connection is connected

Used for tunneling protocols. If the device is disconnected abnormally after the connection is successful, the gateway may not know that the device is disconnected and cannot be automatically reconnected.

3 Security Measures

3.1 Ignore the Retain message

Retain messages are ignored (the MQTT server only marks the real Will message as retain, so it doesn't affect messages sent normally).

3.2 Some commands must be encrypted

System restores, file transfers, background commands, and password changes must be encrypted (i.e., the gateway key must be known).

4 Example of transparent transmission

Transparent transmission support:

Add encodeType to the device configuration, and the value can be hex or string

Point configuration is used to configure the collection command, and paramCommand is added

Configuration example:

{

"protocolCode": "TCP_TUNNEL",

"protocolName": "TCP Tunneling Protocol".

"resolvePropertyMap": {

"ip": "192.168.136.1",

"port": "503",

"rate": "30000" ---------- acquisition interval, each instruction interval is 1 second (later available parameter adjustment), and it takes 1 second to receive data between multiple instructions

"confTabs": [

{

"deviceCode": "deviceTcpTunnel",

"encodeType": "hex",

"resolveParamConfigVOList": [

{

"paramCode": "read" ,---------- arbitrary

"paramName": "Read 10 hold registers from 0" ,--------- arbitrary

"paramCommand": "00 06 00 00 00 06 01 03 00 00 00 0A" ---------- instruction

}

]

}

]

}

Reported data:

(This reported data is not related to the instructions above!) ）

{

"func": "push",

"ts": 1710150390230,

"uid": "270e27cb353752fd4bedc4f1f32d634f",

"identify": 175,

"data": [

{

"deviceCode": "tunnel",

"dataValues": [

{

"paramCode": "tunnel",

"paramName": "tunnel",

"ts": 1710150390230,

"val": "000600000017010314000A109AFFFE0000CDCC0C400000000000000000" ----------- only this one is meaningful, the rest are meaningless, and the encoding is the same as the instruction (the same is hex or string)

}

]

}

]

}

(ENDS)