MQTT: The Protocol Designed for a World Where Things Fail

The origin story of MQTT

Imagine thousands of sensors distributed across a factory, a fleet of delivery drones, or millions of smart meters in different locations. How can these devices communicate reliably when networks are slow, unreliable, and expensive? MQTT (Message Queuing Telemetry Transport) was created to solve exactly this problem.

The origin story of MQTT

The story of MQTT begins in the late 1990s, before the term IoT (Internet of Things) became popular. At that time, engineers were already trying to connect remote machines, sensors, and industrial equipment, but the available communication methods were not well suited for these environments.

Most people describe MQTT as a messaging protocol, but it was created because the physical world is unreliable. Instead of describing it feature by feature, I’ll uncover it through the questions it was designed to answer. Before that, it helps to understand the three roles involved in every MQTT system.

The publisher

A publisher is a device or application that sends information. It publishes messages to a topic without knowing who, if anyone, will receive them.

For example: a temperature sensor may publish the current temperature every minute, or an EV (Electric Vehicle) charger may publish its charging status.

The subscriber

A subscriber is a device or application that receives information. It subscribes to one or more topics and automatically receives every message published on those topics.

For example: a monitoring dashboard can subscribe to temperature readings from all sensors in a factory, while an alarm subscribes to the same data to detect overheating.

The broker

The broker is the central communication hub. Every message sent by a publisher first arrives at the broker, which then forwards it to all subscribers interested in that topic.

The broker also handles many of MQTT’s reliability features, such as keeping track of connected clients, storing messages when needed, and detecting when a client disconnects unexpectedly.

What do clients and brokers look like in practice?

None of this is as abstract as it sounds. A “client” is just anything running MQTT client software: an ESP32 microcontroller with a few kilobytes of firmware, a Raspberry Pi running Home Assistant, a smartphone app checking today’s solar production, or a few lines embedded in an industrial PLC. A commercial smart plug running Tasmota is a client. Home Assistant itself is a client too; it just happens to be one that subscribes to hundreds of topics and publishes commands back to many of them at once.

A “broker” is just as unglamorous: server software, usually a single process. Eclipse Mosquitto is the one most home labs reach for — lightweight enough to run on a Raspberry Pi, in a Docker container, or as a Home Assistant add-on, and largely forget about; it comfortably handles thousands of messages a second on hardware that costs less than the charger it’s monitoring. At the other end of the scale, EMQX and HiveMQ are broker platforms built to run clustered across many servers, handling millions of simultaneously connected clients — the kind of infrastructure a nationwide smart meter rollout would need. Cloud providers also sell brokers as a managed service — AWS IoT Core, Azure IoT Hub — so a manufacturer never has to run one at all.

None of that changes the protocol underneath. A $3 microcontroller and a clustered enterprise broker are speaking exactly the same MQTT to each other. The roles are identical; only the scale differs.

One client, multiple roles

In MQTT, a client is not limited to a single role. The same client can both publish and subscribe.

For example, a smart thermostat publishes the current room temperature while also subscribing to commands that change the desired temperature.

In practice, MQTT has only two types of participants:

Clients, which can publish, subscribe, or do both
The broker, which connects clients and manages the communications

This simple architecture is one of the reasons MQTT scales so well. Publishers never need to know who receives their messages, and subscribers never need to know who produced them. The broker takes care of connecting the two.

Looking a bit closer, MQTT implements the Pub/Sub architecture pattern. It is not something unique to MQTT; MQTT is simply one implementation of that pattern. The key idea behind this is decoupling. Publishers and subscribers do not know about each other. They only agree on a common “language” (topics or events), while a broker or message bus delivers the messages.

MQTT’s Publish/Subscribe model is closely related to several other software patterns, like the Observer Pattern, common in object-oriented programming. An object (the subject) notifies all registered observers when its state changes. You’ll find it most often in GUI frameworks. You can find the same behaviour in Event-Driven Architecture — an event-driven communication pattern applied to the physical world. Instead of software services producing business events, devices produce events about the state of the real world. The architecture is the same: something happens, an event is published, and interested systems react.

Networks fail. Power disappears. Cellular coverage comes and goes. Sensors reboot. Messages get lost. Yet industrial automation, smart cities, connected vehicles and energy systems still need to operate.

How do you know a device is still alive?

Imagine a temperature sensor installed on a remote pipeline. It may not send data every minute because the temperature hasn’t changed. If nothing arrives for an hour, is the temperature stable? Or has the device died?

MQTT answers this with the Keep Alive mechanism.

Rather than continuously transmitting data, the client periodically proves it is still connected. If the broker stops hearing from it within the agreed interval, the connection is considered lost.

What happens when a device suddenly disappears?

A factory robot loses power, a drone flies out of cellular coverage, an EV charger reboots. Nobody sends a goodbye message. How does the rest of the system know?

MQTT introduces the Last Will and Testament (LWT).

Before communication starts, the client leaves instructions with the broker:

“If I disappear unexpectedly, publish this message on my behalf.”

Other clients immediately know that the device went offline.

What happens to messages while the device is offline?

Not every message has the same value. A temperature reading from five minutes ago may be useless. A payment confirmation must never disappear. MQTT allows applications to choose. Should messages be discarded or should the broker keep them until the client reconnects?

This is where persistent sessions become important. Instead of treating disconnections as failures, MQTT treats them as temporary interruptions.

How do you know a message actually arrived?

A sensor publishes a reading. Did the broker receive it? Did the subscriber? On a shaky mobile network, packets get dropped mid-transmission all the time. MQTT doesn’t leave this to chance — it lets you choose, per message, how hard the protocol should work to make sure it lands.

At QoS 0, a message is sent once and forgotten — fire and forget. Fine for a reading that will be superseded a second later anyway.

At QoS 1, the message is redelivered until the receiver acknowledges it — guaranteed to arrive, but possibly more than once.

At QoS 2, MQTT adds an extra handshake that guarantees the message arrives exactly once — no duplicates, no drops — at the cost of more round trips.

Nobody has to pick one level for the whole system. A pipeline pressure reading might travel at QoS 0. A valve-close command might travel at QoS 2. The protocol lets the importance of the message decide how much effort goes into delivering it.

How does a system recover after failure?

A client reconnects after being offline for ten minutes. What happened to it in that time?

If it connected with a persistent session, the broker was holding onto anything sent to it at QoS 1 or QoS 2 while it was gone — those messages arrive the moment it comes back, in order. The client doesn’t need any special recovery logic; from its point of view, the outage just collapses into a slightly longer wait.

If it connected with a clean session instead, none of that history follows it. It starts fresh, subscribes again, and only sees what’s published from that point forward.

Neither choice is “correct” on its own — a factory alarm system wants the messages it missed; a live GPS tracker mostly doesn’t care what happened while it was disconnected, only where the vehicle is now. MQTT lets each client decide which kind of recovery it actually needs.

Why wasn’t HTTP enough?

HTTP is a request-response protocol: a client asks, a server answers. That model assumes the client knows who to ask, and that the server is reachable and listening right at that moment.

Neither assumption holds up well for a sensor network. A pipeline sensor doesn’t know which of several dashboards, alarms, or logging systems might care about its reading — and it shouldn’t have to know. If it had to open a connection and send a request to each one individually, adding a new subscriber would mean reconfiguring every publisher in the system.

HTTP also expects a server to be reachable on demand. Devices on a cellular or satellite link aren’t always reachable, and polling constantly to ask “did anything change?” burns bandwidth and battery for the vast majority of checks that come back with nothing new.

MQTT flips the relationship. A device doesn’t ask anyone anything — it announces what it knows, and whoever’s listening gets notified. Adding a new subscriber means one more client connecting to the broker, not any change on the publisher’s side at all.

Why does MQTT still matter, decades later?

MQTT was designed in 1999 for oil pipelines and satellite links — about as far from a smart home as you can get. And yet it’s the same protocol quietly running inside Home Assistant, inside industrial SCADA systems, inside the EV charger in someone’s garage.

That’s not nostalgia. The problem MQTT solves was never really about the technology of any particular decade — networks were unreliable in 1999 and they’re still unreliable now, just in new shapes: a Wi-Fi dropout instead of a satellite delay, a cheap microcontroller instead of a pipeline modem. The underlying pattern — decouple publishers from subscribers, let a broker absorb the unreliability, treat presence and disconnection as first-class events instead of edge cases — turned out to generalize to almost anything that produces data intermittently and needs that data to reach more than one place.

That’s why a protocol old enough to predate the term “IoT” is still the obvious choice for a new EV charger deciding when to draw power from the sun instead of the grid.

Istvan Dobrentei