MQTT - universal protocol for cloud and IoT applications

In monitoring systems, measured data need to be shared in order to quickly react to changes. There are many monitoring units on the market that can measure any physical quantities. A universal protocol, such as MQTT, is needed when the measured values have to be shared with other devices or third-party systems. MQTT in particular is, thanks to its strengths, nowadays integrated by most producers of advanced IoT devices and applications. To demonstrate these advantage, let us take Poseidon2/Damocles2 devices and a Raspberry Pi as examples.

Poseidon2 family- in addition to measuring temperature, humidity or light intensity, these devices can also detect flood or absence of water, measure air flow, and more. They can send e-mail notifications and upload the measured values to a computer or to the dedicated SensDesk portal, where the data is displayed in easy-to-understand charts accessible from any computer, tablet or mobile phone over the Internet.

Damocles2 family devices provide lots of digital inputs for connecting dry contacts (e.g. 24 inputs in the 2404 model). All digital inputs feature S0 pulse counters with a memory to connect water, gas, electricity or other meters. Digital inputs can also monitor the opening/closing of doors or windows. For instance, a slightly open freezer door can cause lots of problems. When combined with a motion detector, the system can also alert to unwanted visitors. Built-in pulse counters help provide an overview of the total energy consumption at the monitored premises.

 

Both product families can share the measured values over MQTT. MQTT compatibility allows connecting to IoT Hub, MS Azure, AWS IoT, Bluemix Internet of Things and other cloud services and third-party systems.

MQTT: Basic idea

In IoT systems, the sensors should not decide what to do. Sensors should simply send the results of their work to a central location, and nothing more. Whoever needs these messages then subscribes to them, can process them and respond. For example, a second layer can be added where a relay reacts to switch on/switch off requests. A switch can notify that it has been switched, a temperature sensor can inform about the measured temperature, and the central location forwards all this information to the respective subscribers. This makes it easy to react to changing application requirements and to add functions, such as switching a relay with a delay or remapping physical devices to work in a more logical way.

 

History of MQTT

At the beginning, when thinking about a protocol to use for communication, HTTP was a unanimous choice. Everyone knows it and it is common world-wide. However, it became apparent that HTTP had many disadvantages for the Internet of Things. There is a lot of communication and the implementation is complicated. Moreover, the HTTP protocol is based on the “request-response” paradigm; so, various workarounds, such as long polling, heartbeat, HTTP/2, and so on, are needed for server-initiated communication. There are no QoS levels, and two HTTP devices always need a server application to enable mutual communication because a HTTP server alone cannot pass data between clients.  For these reasons, a binary version - CoAP - was developed to simplify implementations in MCUs. It reduced the data volume by replacing text headers with binary flags. However, with optional extensions, CoAP also become bloated, losing its main advantage.

For these reasons, the MQTT protocol (formerly: Message Queuing Telemetry Transport, now: MQ Telemetry Transport) was created. It is a simple and lightweight protocol for passing messages between clients through a central point – a broker. Thanks to these characteristics, it can be easily implemented even in devices with simple microcontrollers, and quickly became popular. It was designed by IBM and is currently maintained by the Eclipse foundation. Recently it has become an OASIS standard.

In principle, MQTT is based on the exchange of messages between clients through a central server – the MQTT broker. Messages are transferred over TCP using a Publisher – Subscribers structure. Messages are divided into “topics”; a device may either “publish” a given topic, i.e. send data to the MQTT broker, or “subscribe” to the topic to receive the data.

Theoretically, a client can be a publisher and a subscriber at the same time; however, these functions are usually separated. A publisher is usually a sensor or a measuring device that sends the measured values to the MQTT broker, while a subscriber is usually a control unit that receives (subscribes to) the values for further processing or visualization.

Message content

The message content is not defined in any way. It is simply any binary data that need to be passed on. The most common data formats include JSON (JavaScript Object Notation), BSON (Binary JSON) or plain text. Still, the format is not restricted in any way since MQTT does not process the data, only passes them on. It is solely up to the subscriber to “understand” the data. In the current protocol version, the message size is limited to a little less than 256 MB; however, because of the focus on “Internet of Things”, most messages are much shorter.  One of the aims of MQTT is to minimize the data volume. There is only a minimum of overhead.  MQTT is therefore suitable for occasional transfers of information and values, where transfer speed is not critical – ideal for IoT purposes.

Data transfer model

The MQTT protocol by itself only specifies the message structure but does not define the data transfer method. For this purpose, it uses the common TCP/IP protocol over a common Ethernet LAN or over the Internet. MQTT is at the application layer of the OSI model. Thanks to the simple structure and the use of the common Ethernet interface, MQTT can be easily implemented in devices with low-power, low-performance processors and is quickly becoming widespread. For the same reasons, it is quickly becoming popular in industry; MQTT can be relatively easily implemented into PLCs already equipped with a TCP/IP interface (Ethernet and RJ-45 jack) without any modification of the CPU unit hardware. 

For the message transfer, MQTT defines three QoS (Quality of Service) levels of message acknowledging:

• The lowest “QoS 0” level means that a message is sent without an acknowledgement and delivery is not guaranteed (at-most-once).
• The middle “QoS 1” level means that the message is delivered at least once.
• The highest “QoS 2” level means that every message is delivered exactly once.

However, a client need not support all three QoS levels; the MQTT protocol does not prescribe that. 

Security

The MQTT protocol by itself does not enforce or require any security. It is up to the client or device manager to decide whether or not to implement encryption. Therefore, it is advisable to be careful with public brokers and applications, especially from suspicious or unknown sources. When a public broker is used for the data, there is no effective control over other brokers that might, even temporarily, connect to it.

Security is generally ensured in closed networks or IT systems. To secure the transferred content, MQTT offers several options. Each communication between the client and the broker begins with the client logging in (“CONNECT” mode). In the login sequence, the client is identified with its “ClientID” and optionally with a “Username” and a “Password”. If the client capabilities allow, it is suitable to set the “ClientID” to a simple string that identifies the device well and helps keep an overview about the network.

Thanks to SSL/TLS support, MQTT supports logging in using a client SSL certificate. Supported protocols include TLS v1.2, v1.1 or v1.0 with x509 certificates. It is important to keep in mind that the MQTT protocol is purely text-based; without SSL/TLS, the communication is completely open (and the password is the primary concern).

Depending on the desired security level, the MQTT protocol prescribes the following TCP channels:

• 1883 = unencrypted MQTT - basic and most common MQTT communication channel; the communication is not encrypted in any way (including the password, which is sent in the open). This channel should not be used for any sensitive data.
• 8883 = encrypted MQTT - unlike the 1883 channel, data are encrypted using the SSL/TLS protocol, and client-side support is needed to establish the connection.
• 8884 = encrypted MQTT + client certificate - this is the highest level of security available for MQTT communication; in addition to the data being encrypted using the SSL/TLS protocol, the client must authenticate itself with a certificate issued by the broker. However, so far this channel is only supported by a handful of public brokers (e.g. Mosquitto - “test.mosquitto.org” server).

Simple MQTT setup

Poseidon2 and Damocles2 devices (external link) already have this protocol implemented and the setup requires no programming experience. Simply enable MQTT support in the device, set the MQTT broker’s IP address, username and password, and select the connected sensors to publish to the MQTT broker. The previous description makes it clear that a topic to publish (and to subscribe to at the other end) needs to be selected. A MQTT broker can be chosen from this list: https://github.com/mqtt/mqtt.github.io/wiki/public_brokers (external link). With a single click, the communication can be secured with SSL.

Sharing data with other devices and systems

MQTT can send data to IoT Hub, MS Azure, AWS IoT, Bluemix Internet of Things and other clouds. The setup is very simple, see this article for a description.

MQTT ensures interoperability of otherwise very different devices. Thanks to MQTT’s popularity, numerous libraries are available to facilitate the implementation in various processors. For example, Digikey offers instructions on how to implement MQTT in Raspberry PI 3 with ESP8266. The combination of HW Group products with Raspberry Pi is an effective system where Poseidon2 and Damocles2 take care of sending sensor data to the broker, and the Raspberry Pi reads, processes, and acts upon the values.  The result is a professional application at the measurement side, and the customer can focus on the application part.