MQTT
What is MQTT?
In the most simple terms, MQTT is a messaging protocol that was designed to create a reliable standard for machine-to-machine (m2m) communication. IBM’s Andy Stanford Clark and Eurotech’s Arlen Nipper created the protocol in 1999.
MQTT is a publish-and-subscribe protocol, meaning that instead of communicating with a server, client devices and applications publish and subscribe to topics handled by a broker.
MQTT typically uses IP (Internet Protocol) as its transport but can also use other bi-directional transports.
What is MQTT-SN?
MQTT-SN (MQTT for sensor networks) is a variant of MQTT that has been optimized for use in low power environments such as sensor networks, as the name suggests.
On top of the standard feature set, SN adds extra functionality for use cases where lower power is required. These extra features include:
-
-
- QoS mode -1: allows for fire-and-forget messaging
- Topic aliases: allows for simplified publishing and reduced data overheads
- Sleep mode (disconnected sessions): allows messages to be queued on the broker while the remote Thing or device is powered off
-
Why MQTT is perfect for IoT
When you boil it down, the Internet of Things (IoT) has one job: to get data to and from devices on a network. The problem is that these networks could be anywhere in the world and each faces many conditions that can and will cause them to fail. MQTT, and by extension MQTT-SN, has myriad features built in to help mitigate some of these issues. Some of the main features are listed below:
1. MQTT is easy
Getting up and running with MQTT is quick and easy. There are millions of ready-made client applications and almost as many brokers available. You can get started with MQTT right now by using the broker integral to the u‑blox Thingstream platform.
2. Reliability of MQTT
Many IoT devices rely on a radio connection to transmit and collected data, meaning connectivity isn’t always reliable. MQTT can allow for messages to be stored at the broker until a device is ready to receive it. Thanks to QoS (Quality of Service), MQTT has the ability to queue messages, make sure they get where they are going and if required, ensure that they only get there once.
3. Bidirectional messaging
It is perhaps more accurate to say that MQTT is omnidirectional. Any device, Thing, or application can publish or subscribe to any topic handled by the MQTT broker. This means there is no limit on what can talk or listen to what across the network.
4. Messaging at scale
With MQTT, it is just as easy to broadcast a message to a million devices as it is to send to just a hundred. To be heard by everything on the network, a Thing simply needs to publish to a topic to which all devices are subscribed.
What are the advantages of MQTT?
MQTT client and broker
Don’t think, “client and server”, think, “client and broker” instead. In a traditional client/server relationship, the client and the server connect, and the server is treated as a storage and distribution depot for the data. With MQTT, the process is different. The MQTT broker is much more passive, acting more like a signpost for where the data should go.
MQTT client
Any Thing (from a microcontroller to a massive server) that runs an MQTT library and connects to a broker over a network can effectively become an MQTT client.
Clients don’t send messages directly to and from each other but instead communicate to topics managed by the MQTT broker. These topics work a little like email inboxes. Messages are published by Things to topics; messages are then picked up when a Thing subscribes to those topics.
MQTT broker
The MQTT broker handles authentication of Things on the network as well as managing connections, sessions, and subscriptions. Its main responsibility is to receive all published messages and then send them subscribed clients. The MQTT broker also queues messages for subscribed clients, delivering them according to the agreed QoS level.
MQTT devices and applications
In an MQTT network, MQTT devices and applications are usually known as “things”. The reason for this is that there is no difference between the two as far as the broker is concerned. To that end, both devices and applications can publish and subscribe to topics managed by the broker.
MQTT devices
There are various types of MQTT-enabled devices in the field today, ranging from simple Arduino-based devices to devices for mission-critical commercial, industrial, and medical applications. Many smart homes and businesses are also built around interconnected MQTT devices.
Instant IoT
For businesses looking to launch digital transformation or IoT trials, ready-made MQTT-based devices are available from u‑blox. These devices come with connectivity baked-in and all of the complicated technical work taken care of. These devices utilize the u‑blox global IoT service, MQTT Anywhere, meaning that they will work in over 190 countries worldwide.
More information about ready-made IoT solutions:
MQTT topics
MQTT messages aren’t delivered directly from Thing to Thing. Instead, they are published to “topics”. The broker then delivers those messages to any subscribed clients.
Anatomy of an MQTT topic
-
-
- Topics are made up of one or more topic levels, separated by a forward slash:
-
-
-
- Topics are case sensitive
- Topics don’t have to be pre-registered at the broker.
-
MQTT-SN topic features
MQTT-SN adds some special topic features to assist in a bandwidth-constrained environment:
-
-
- If using predefined topics, you can create a topic alias on the broker which the client can then use without the need to register first. This can reduce the number of billable messages.
- The long topic name does not have to be sent over the air in either direction. This saves bandwidth, and there is no need to store the long topic name in memory.
- If not using predefined topics, Things use the REGISTER command to register a topic name with the server. The server will respond with a REGACK containing a topic ID consisting of 2 characters. Note that you would need to register the topic each time the client connects.
-
Why use MQTT topics?
Topics are a great way to organize data as it flows through the network and this becomes more apparent with scale. For example, if you have devices with several sensors deployed across multiple sites, you could put all of the data in one payload and parse it when it gets to its destination or you could do it the MQTT way and use topics to divide up the data, as shown below:
-
-
- site1/position
- site1/temp
- site1/vibration
- site2/position
- site2/temp
- site2/vibration
-
When the data transmitted is divided by topic, Things can then subscribe to the topics they are interested in. If a device is interested in multiple topics, they can be subscribed to individually or wildcards can be used. For example, to get all the data from site1, you would use the wildcard, “site1/#”. You could also get all “temp” data from all sites by using the wildcard, “+/temp”.
Connections
Connections are always made between a client and a broker. Clients cannot connect directly to each other.
The connection is established by the client sending a CONNECT message, to which the broker responds with a CONNACK (connection acknowledged). A connection is required in order to publish with a Quality of Service (QoS) level 0, 1, or 2 or to subscribe to any topics. Clients usually connect to a broker using a Client ID (ClientID), username, and password. A client may only have one connection with the same broker at any given time.
Clean session
The “clean session” setting gives you the ability to start fresh with no messages in the queue.
-
-
- false: The broker stores all subscriptions for the client and will queue any messages for that subscription that were published with a QoS level 1 or 2
- true: The broker purges all queued messages when the client connects.
-
Keep alive
The “keep alive” setting defines the longest period of time that the connection can remain in place without the broker or client sending a message. This allows battery-operated to go into a sleep state during which all messages destined to them are buffered at the server/gateway and delivered later to them when they wake up.
To keep the connection in place, the client sends a PINGREQ before the keep alive window expires, to which the broker responds with a PINGRESP. When the device has not been seen within the keep alive window, the broker can choose to disconnect the session, and will then stop sending messages destined to the device until the next session.
Sleep (MQTT-SN only)
An MQTT-SN client can tell the broker it is going to sleep for a period of time by sending a DISCONNECT with a DURATION greater than 0. The broker will maintain this state for the period specified. The broker will then queue all subscribed messages (Qos 0, QoS 1, or QoS 2) in the sleep state. This differs from a being disconnected with “clean session” set to false, which only retains QoS 1 & QoS 2 messages.
Whilst asleep, the client can flush the queue without needing to wake up by issuing a PINGREQ. The broker will then respond with a PUBLISH if there are messages to deliver, and a PINGRESP when the flush is complete, putting the device back to sleep.
Subscriptions
MQTT clients don’t connect directly to each other, instead, they subscribe to topics to receive messages.
To subscribe to a topic, the client must first send a SUBSCRIBE request to the broker. The SUBSCRIBE request can include multiple topics. The broker responds to the SUBSCRIBE request with a SUBACK (subscription acknowledged) response. Subscriptions also have a QoS setting which can be used to downgrade the QoS of the published message. In this case, the message is always published at the lower QoS setting.
Subscription wildcards
Subscriptions can use one of two types of wildcard shown below.
Single-level (+)
A single-level wildcard replaces one topic level. As shown in the image below.
This wildcard would cover the following topics:
-
-
- sensors/soil/out
- sensors/water/out
- sensors/light/out
-
Multi-level (#)
A multi-level wildcard replaces multiple topic levels:
This wildcard would cover the following topics:
-
-
- sensors/soil/out
- sensors/soil/in
- sensors/temperature/out
-
MQTT QoS explained
Where the MQTT protocol is concerned, the guarantee of delivery is defined by QoS (Quality of Service).
Here, you will learn how, where, and when to use MQTT QoS, and which levels are right for your own IoT applications.
MQTT QoS levels
MQTT and MQTT-SN support multiple levels of QoS for guaranteeing message delivery.
QoS -1 – fire and forget
QoS -1 (minus one) is ideal for low-power non-critical applications where it doesn’t matter if every message gets to where it’s going. By not making a hard connection with the broker and receiving no acknowledgement, considerably less power is used to complete the transaction.
QoS -1 key features:
-
-
- Only available for devices using MQTT-SN
- Does not require an MQTT connection to be established
- No acknowledgment from the recipient
- Not retried by the sender
- Analogous to QoS 0 by the time it reaches the broker
-
When to use QoS -1:
-
-
- Ideal for power-constrained Things to minimize time on air
- Minimize messaging cost
- OK if message delivery is not critical e.g. data sent frequently
-
QoS 0 – at most once
QoS 0 (zero) is used to ensure that a message reaches its destination no more than once. Unlike QoS -1, this method requires an MQTT connection meaning it is less efficient in terms of power.
QoS 0 key features:
-
-
- Best effort message delivery
- No acknowledgement from the recipient
- Not retried by the sender
- Not queued by the broker for disconnected clients with a valid subscription to the topic
-
When to use QoS 0:
-
-
- Good for power-constrained Things to minimize time on air
- Minimize messaging cost
- OK if message delivery is not critical e.g. data sent frequently
- Not as efficient as MQTT-SN QoS -1 due to the requirement of MQTT CONNECT
-
QoS 1 – at least once
QoS 1 is used when message delivery is critical. This is achieved by queueing messages until the subscriber is able to receive it.
QoS 1 key features:
-
-
- Guarantees that a message is delivered at least one time to the recipient.
- Sender stores the message until it receives a PUBACK from the recipient
- Messages may be sent or delivered multiple times.
-
When to use QoS 1:
-
-
- Use when you have to receive every message, but make sure you handle duplicates
- Use if you want messages to be queued on the broker for delivery to offline Clients
- Use when the overhead of QoS 2 is too high
-
QoS 2 – exactly once
QoS 2 is used when the message needs to arrive once and only once. This level is used when delivery is essential.
-
-
- QoS 2 is the safest and slowest Quality of Service level
- Guarantees that each message is received only once by the intended recipients by using at least two request/response flows (a four-part handshake) between the sender and the receiver.
-
When to use QoS 2:
-
-
- Use if message delivery is critical and duplicate data is harmful to subscribers
-
QoS downgrade
In cases where there are many devices on the network, different levels of QoS might be needed. To achieve this, MQTT allows downgrading of the QoS level at the subscriber node. The result of this is that the QoS for a message that is published does not have to be the same as the QoS for a message that is received.
QoS is defined by the original publisher, but when the broker then delivers the message on to subscribers, the lower of the PUBLISH QoS and the QoS defined during the SUBSCRIBE is used.
Security – is MQTT secure?
Making sure IoT devices are secure is just as important as making sure that they work and every link or node on the network is a potential exploit vector. While MQTT itself does contain some security mechanisms, it’s important to consider factors extraneous to the transport itself.
Network-level security
The first place to consider security is the network itself. Ensuring that the network connection is secured by using a VPN tunnel will prevent exposure to network traffic.
Transport-level security
As with HTTP traffic, MQTT traffic can be secured at the transport layer with TLS/SSL.
Application-level security
Unique client identification and username/password credentials are provided by the protocol itself and should be considered the bare minimum security requirement.
Payload encryption
Further security can be added by encrypting the payload itself at the application level.
U-blox Thingstream MQTT security
The u‑blox Thingstream service delivery platform adds an extra layer of security on top of those mentioned above by not exposing the IP address of the device. By having no physical connection to the Internet, this makes devices much harder to exploit.
MQTT v5.0 and future evolution
In 2019, standards body, OASIS released the official MQTT 5.0 standard.
New features added to the MQTT Version 5.0 standard:
-
-
- Reason codes: Acknowledgements now support return codes, which provide a reason for failure.
- Shared subscriptions: Allow the load to be balanced across clients and thus reduce the risk of load problems
- Message expiry: Messages can include an expiry date and are deleted if they are not delivered within this time period.
- Topic alias: The name of a topic can be replaced with a single number
-