Popular messaging protocols
Comparing the overhead in constrained IoT networks
Introduction
When it comes to constrained Cellular IoT network deployments, the choice of the application-layer protocol to interconnect end-devices with a cloud infrastructure is critical, as it directly impacts the cost to convey data. i.e. the amount of data volume consumed in nodes on a monthly basis. The data volume also affects the energy performance, i.e. how long the nodes can autonomously operate using batteries, which is relevant to both cellular and non-cellular IP IoT technologies (e.g. WiFi).
In constrained IoT deployments, the end-devices are usually built upon resource- and/or energy-constrained hardware and communicate with the cloud through a bandwidth-limited and expensive radio technology. Typically the amount of useful information, i.e. the payload, is a few bytes, transmitted sporadically, for example once per hour or even once per day. In IP-based networks, like NB-IoT, and contrary to LoRaWAN, Sigfox or other sub-GHz proprietary technologies, transport of data is facilitated through specific protocols implemented upon the IP stack, resorting to an increase in overhead.
This article provides a quantitative analysis of this protocol overhead, using data from real traffic traces captured during emulated IoT communication scenarios. For this purpose, and towards representing realistic end-to-end (from device to cloud) IoT interconnection, we also use an IoT platform implementation.
Messaging Protocols
MQTT and CoAP are today’s most popular messaging protocols for IoT applications, thanks to their simplicity, low-bandwidth requirements, lightweight/small code base implementation. In summary:
- MQTT (Message Queue Telemetry Transport) is an open OASIS and ISO standardized lightweight, publish-subscribe network protocol that transports messages between devices and running over TCP/IP. Its origins go back to 1999, where the first IBM-driven version was created for monitoring an oil pipeline through the desert. The goal was to have a protocol that is bandwidth-efficient, lightweight and uses little battery power, because the devices were connected via expensive satellite links. A variant of MQTT, MQTT-SN, was later introduced for use over other transports such as UDP/IP, Bluetooth, and Zigbee.
- CoAP (Constrained Application Protocol) is a specialized Internet Application Protocol defined in IETF RFC 7252, and running over UDP/IP. CoAP is used with constrained nodes and constrained networks. The protocol is designed for machine-to-machine (M2M) applications such as smart energy and building automation.
For smart-home/smart-industry applications where a local WiFi network provides inexpensive Internet connectivity there is no real concern on the utilization of these protocols. On the contrary, in remote locations where Internet connectivity could be provided by NB-IoT under very constrained data plans (e.g. 500 KB/1MB per month/device), accurate prediction of overhead at application-level is required.
Experimental Setup & Tools
We will evaluate the protocol overhead in a typical end-to-end IoT application setup, focusing on the end-node to cloud communication direction. All experiment components are deployed in a single standard Ubuntu host for convenience:
- To emulate the client-side of the messaging protocols, we use standard Linux binaries or open-source implementations available in GitHub. In particular:
- For MQTT we use the
mosquitto_pubbinary coming from the standard Eclipse Mosquitto clients package (mosquitto-clients) - For MQTT-SN we use the python-based library from https://github.com/ehong-tl/mqttsn-python3-micropython.git
- For CoAP we use the
coap-cli, a Go language implementation, taken from the open-source project https://github.com/mainflux/coap-cli
- For MQTT we use the
- To emulate the cloud-side we use a free-to-use, open-source IoT platform called https://github.com/mainflux/mainflux. Mainflux is a scalable, secure, open-source, and patent-free IoT stack written in Go. It accepts user and thing connections over various network protocols, including MQTT and CoAP, thus making a seamless bridge between them. It is used as the IoT middleware for building complex IoT solutions. The basic operation principle of Maiflux is the following: Each IoT end-node is associated with an
id/keytuple and through it is allowed to access achannelfor carrying an IoT payload, formatted using recent IETF’sSenMLstandard. Data upon arrival is stored in a time-series influx database. The Mainflux stack provides MQTT and CoAP adapters for listening to published MQTT messages and CoAP POST messages from various end-points. For MQTT-SN an intermediate transparent gateway is needed to “translate” MQTT-SN messages to standard MQTT messages. For this purpose we use the open-source implementation by Eclipse Paho, available in https://github.com/eclipse/paho.mqtt-sn.embedded-c/tree/master/MQTTSNGateway - To capture and analyze the traffic exchanged between the emulated IoT end-devices and the Mainflux IoT platform we use the Wireshark tool, which comes with readily available packet dissectors for IP layer, Transport Layers (TCP/UDP), and Messaging Layers (MQTT, MQTT-SN, CoAP).
Protocol Overhead Analysis
Mainflux Thing: id: 244629b1-389e-4b33-8a82-da49368e0d40 key: 06d14578-989a-4580-b1cd-832fd2ee8da6Along the same lines, a test IoT “channel” has been created for consuming the data and writing them to the Mainflux’s influx instance. The specific channel has been associated with the above “thing”:
Maiflux Channel: id: 8f3de729-6a42-48d5-a266-20db9c7bda35Finally, an example IoT payload is generated considering a simple message conveying a single temperature reading (20 degrees of Celsius), formatted as a SenML message (“bn” stands for the base name that differentiates the specific device from others):
IoT Payload:
'[{"bn":"testdev-","n":"temp","u":"C","v":20.0}]'
MQTT
To send the required message using the standard MQTT protocol we issue the following command in a Linux terminal:
<br />mosquitto_pub -h localhost -p 1884 \<br />-u 244629b1-389e-4b33-8a82-da49368e0d40 -P 06d14578-989a-4580-b1cd-832fd2ee8da6 \<br />-t 'channels/8f3de729-6a42-48d5-a266-20db9c7bda35/messages/244629b1-389e-4b33-8a82-da49368e0d40/' \<br />-m '[{"bn":"testdev-","n":"temp","u":"C","v":20.0}]'<br />Notice that authentication is facilitated through the thing’s id and key attributes, passed as MQTT username and password respectively. The configured port (1884) is also different from the default MQTT port (1883). If we use the default MQTT broker setup, then the -p option should reflect 1883.
The MQTT topic name is also determined based on the thing id and the channel id. A record similar to the following one should appear on the influx instance, if MQTT communication succeeds:
<br />name: messages<br />--------------<br />time channel link name protocol publisher subtopic unit updateTime value<br />1583687354000000000 8f3de729-6a42-48d5-a266-20db9c7bda35 testdev-temp mqtt 244629b1-389e-4b33-8a82-da49368e0d40 244629b1-389e-4b33-8a82-da49368e0d40 C 0 20</p><p>
Prior to issuing the command we assume that we have started capturing the traffic using Wireshark. We have configured Wireshark to listen to lo interface using the capture filter tcp port 1884. The complete message exchange is shown below (we also attach the corresponding traffic dump in the form of a PCAP file):
A complete MQTT message publishing procedure involves the following steps/traffic exchange:
- Since the MQTT protocol relies on TCP, a new TCP connection must be established at the beginning of each MQTT session. This is a 3-way handshake involving both ends (packets #1-#3). In other words, within each MQTT session an underlying TCP connection is present.
- TCP connection establishment is followed by an MQTT connection establishment procedure, involving:
- an MQTT Connect message sent by the client (#4) and the respective TCP ACK message sent by the broker’s TCP end-point (#5)
- an MQTT Connect ACK message sent by the MQTT broker (#6) and the respective TCP ACK message sent by the client’s TCP end-point (#7)
- MQTT connection establishment is followed by the MQTT PUB procedure, which includes the MQTT communication credentials, the topic and the IoT payload information (#8)
- MQTT message publication is followed by the release of the MQTT connection, through the MQTT Disconnect Message sent by the client’s MQTT end-point (#9).
- Following the FIN/ACK flags included in the TCP part of the MQTT disconnect command, the underlying TCP connection is released trough a bi-directional TCP messages exchange (#10-#12).
The following Table shows the exact overhead per message, including the MQTT, TCP and IP overheads. Notice that the payload in our experiments (the [{"bn":"testdev-","n":"temp","u":"C","v":20.0}] string message) is 47-bytes long.
| PROCEDURE | MQTT OVERHEAD (BYTES) | TCP OVERHEAD (BYTES) | IP OVERHEAD (BYTES) | TOTAL OVERHEAD (BYTES) |
|---|---|---|---|---|
| TCP connection establishment (#1-#3) | – | 40+40+32=112 | 3×20=60 | 172 |
| MQTT connection establishment (#4) | 115* | 32 | 20 | 167 |
| MQTT connection establishment TCP ACK (#5) | – | 32 | 20 | 52 |
| MQTT connection ACK (#6) | 4 | 32 | 20 | 56 |
| MQTT connection ACK TCP ACK (#7) | – | 32 | 20 | 52 |
| MQTT message publication (#8) | 144 | 32 | 20 | 196 |
| MQTT disconnection (#9) | 2 | 32 | 20 | 54 |
| TCP connection release (#10-#12) | – | 3×32=96 | 3×20=60 | 156 |
| Total overhead | 265 | 400 | 240 | 905 |
* The MQTT connection establishment message is that large because it also includes the credentials needed to access the Mainflux platform (72 bytes, in particular).
CoAP
To send the required message using the standard CoAP protocol we issue the following command on a Linux terminal:
<br />./coap-cli post \<br />coap://localhost/channels/8f3de729-6a42-48d5-a266-20db9c7bda35/messages/subtopic\?authorization=06d14578-989a-4580-b1cd-832fd2ee8da6 \<br />-d '[{"bn":"testdev-","n":"temp","u":"C","v":20.0}]'<br />Notice, that the authentication in this case is facilitated through the thing’s id passed in the authentication field. We also assume that the default CoAP port (5683) is used. An entry like the following one should appear on the influx instance, if everything goes smoothly:
<br />name: messages<br />--------------<br />time channel link name protocol publisher subtopic unit updateTime value<br />1583688223000000000 8f3de729-6a42-48d5-a266-20db9c7bda35 testdev-temp coap 244629b1-389e-4b33-8a82-da49368e0d40 subtopic C 0 20<br />
Similarly to the MQTT case, we have configured Wireshark to listen to the lo interface, but now we are using the capture filter udp port 5683. The complete message exchange is shown below (we also attach the corresponding traffic dump in the form of a PCAP file):
Since CoAP is based on the UDP/IP transport protocol, there is no need for connection establishment/release and TCP acknowledgements. Hence, the CoAP POST command is encapsulated in a single UDP/IP message (packet #1). In the capture we also view a CoAP ACK message which is used to notify the client about successful message transmission. This is an optional feature of CoAP, so such a 2-way communication is not mandatory. The following table shows the overhead for each layer:
| PROCEDURE | COAP OVERHEAD (BYTES) | UDP OVERHEAD (BYTES) | IP OVERHEAD (BYTES) | TOTAL OVERHEAD (BYTES) |
|---|---|---|---|---|
| CoAP POST (#1) | 183 | 8 | 20 | 211 |
| CoAP ACK (#2) (*optional) | 6* | 8* | 20* | 34* |
| Total overhead without Confirmed CoAP | 183 | 8 | 20 | 211 |
| Total overhead with Confirmed CoAP | 189 | 16 | 40 | 245 |
MQTT-SN
MQTT-SN stands “in-between” MQTT and CoAP, since it borrows the 2-way communication nature of the TCP-based MQTT protocol, but at the same time uses UDP/IP as the underlying transport mechanism. So, we expect some overhead savings due UDP usage. At the same time, there are 2 additional distinctive features of MQTT-SN that enable further overhead savings, compared to standard MQTT:
- Recall that in standard MQTT, each time the client needs to publish something, the full topic name should be included in each message. This could be a long name, consuming a significant amount of bytes. In our example, the topic name is
channels/8f3de729-6a42-48d5-a266-20db9c7bda35/messages/244629b1-389e-4b33-8a82-da49368e0d40/which is 92-bytes long. MQTT-SN introduces “topic registration”, where the long topic name could be mapped to a 2-byte integer, and afterwards this 2-byte field could be used in each publish message, instead of the whole string representation. - In standard MQTT when the client is put to deep sleep (for energy consumption purposes) the one end of the TCP connection fails and the whole session is broken. Hence, after the node wakes up, the session needs to be restored from scratch. MQTT-SN introduces the asleep mode, during which the connection stays active. To achieve this, in MQTT-SN, the client does not establish an end-to-end TCP connection with the MQTT broker. Instead, an intermediate entity called the MQTT-SN Gateway, is responsible for translating UDP packets arriving from/destined to the client to MQTT packets destined to/arriving from the MQTT broker.
Using the open-source MQTT-SN client and gateway implementations reported previously we implemented two scenarios:
- one where a client is supposed to first enter an IoT network, thus a topic registration is also needed, and
- another one where the client has already registered the topic of interest and wakes up from deep sleep for a new transmission.
The traffic captures are provided below. Wireshark is configured with the following capture filter udp port 10000, since we have set the MQTT-SN Gateway to operate on port 10000.
The traffic capture, for the case where the client performs an initial communication with the IoT platform, and thus needs to register the topic, is shown below:
For the case when the topic has been already registered the following traffic exchange is captured:
The corresponding dumps are also provided:
The traffic analysis for the initial scenario is as follows:
| PROCEDURE | MQTT-SN OVERHEAD (BYTES) | UDP OVERHEAD (BYTES) | IP OVERHEAD (BYTES) | TOTAL OVERHEAD (BYTES) |
|---|---|---|---|---|
| MQTT-SN connection establishment (#1) | 19 | 8 | 20 | 47 |
| MQTT-SN connection ACK (#2) | 3 | 8 | 20 | 31 |
| MQTT-SN Topic Registration (* needed only once) (#3) | 98 | 8 | 20 | 126 |
| MQTT-SN Topic Registration ACK (* needed only once) (#4) | 7 | 8 | 20 | 35 |
| MQTT-SN Message Publication (#5) | 54 | 8 | 20 | 82 |
| MQTT-SN Disconnect to Sleep Mode (#6) | 4 | 8 | 20 | 32 |
| MQTT-SN Disconnect to Sleep Mode ACK (#7) | 2 | 8 | 20 | 30 |
| Total overhead | 187 | 56 | 140 | 383 |
whereas for the second scenario is:
| PROCEDURE | MQTT-SN OVERHEAD (BYTES) | UDP OVERHEAD (BYTES) | IP OVERHEAD (BYTES) | TOTAL OVERHEAD (BYTES) |
|---|---|---|---|---|
| MQTT-SN (re-)connection establishment (#1) | 19 | 8 | 20 | 47 |
| MQTT-SN (re-)connection ACK (#2) | 3 | 8 | 20 | 31 |
| MQTT-SN Message Publication (#3) | 54 | 8 | 20 | 82 |
| MQTT-SN Disconnect to Sleep Mode (#4) | 4 | 8 | 20 | 32 |
| MQTT-SN Disconnect to Sleep Mode ACK (#5) | 2 | 8 | 20 | 30 |
| Total overhead | 82 | 40 | 100 | 222 |
Overview
- IoT Payload: 47 bytes
- NB-IoT monthly data plan: 500 KB
| PROTOCOL | OVERHEAD (BYTES) | TOTAL ALLOWED MESSAGES PER DAY |
|---|---|---|
| MQTT | 905 | 18 |
| CoAP | 211 | 80 |
| MQTT-SN First Connection | 383 | 43 |
| MQTT-SN Communication after first connection | 222 | 75 |
