MQTT
Overview
The Airbyte MQTT destination syncs your data to any MQTT system compliant with version 3.1.X. It writes each stream to its corresponding MQTT topic.
Prerequisites
- If you use Airbyte Open Source with the Postgres source connector, upgrade your Airbyte platform to version
v0.40.0-alphaor newer. Then upgrade your MQTT connector to the latest version.
Sync overview
Output schema
The connector outputs each stream to an MQTT topic.
The connector writes data in JSON format as bytes.
Each record’s payload contains these four fields:
_airbyte_ab_id: a UUID that Airbyte assigns to each processed event._airbyte_emitted_at: a timestamp showing when the event was pulled from the data source._airbyte_data: a JSON blob containing the event data._airbyte_stream: the name of the record’s stream.
Features
| Feature | Supported?(Yes/No) | Notes |
|---|---|---|
| Full Refresh Sync | No | |
| Incremental - Append Sync | Yes | |
| Incremental - Append + Deduped | No | |
| Namespaces | Yes |
Get started
Requirements
To use the MQTT destination, you need:
- An MQTT broker that implements MQTT protocol version 3.1.X.
Setup guide
Network access
Make sure Airbyte can access your MQTT broker.
Permissions
Allow Airbyte to write messages to topics. Check whether your deployed MQTT broker requires specific permissions.
Target topics
Use the topic_pattern configuration parameter to choose where messages are written. You can specify one predefined topic or set topics dynamically using the source namespace or stream.
To write every message to one predefined topic, enter its name in the topic_pattern field. For example, setting topic_pattern to path1/path2/my-topic-name sends messages from every stream and namespace to that topic.
To define output topics dynamically, use the {namespace} and {stream} pattern variables. These variables route messages to different topics using the values available when records are produced. For example, set the topic_pattern parameter to airbyte_syncs/{namespace}/{stream}. Messages from namespace n1 and stream s1 are then written to topic airbyte_syncs/n1/s1. Messages from s2 are written to airbyte_syncs/n1/s2, and so on.
Set up the MQTT destination
You can now configure MQTT as a destination in the Airbyte UI. The following parameters are available, although many are optional or have default values:
- MQTT broker host
- MQTT broker port
- Use TLS
- Username
- Password
- Topic pattern
- Test topic
- Client ID
- Sync publisher
- Connect timeout
- Automatic reconnect
- Clean session
- Message retained
- Message QoS
For more information, see OASIS MQTT standard site.
NOTE: MQTT version 5 isn’t supported yet.