Advanced Analytica Advanced Analytica: IBOM
← All connectors
MongoDB logo

Destination connector

MongoDB

Send governed data into MongoDB with a clear, practical setup guide.

Version 0.2.0 alpha

MongoDB

Prerequisites

  • If you use Airbyte Open Source with the Postgres source connector, upgrade your Airbyte platform to version v0.40.0-alpha or newer. Upgrade your MongoDB connector to version 0.1.6 or newer.

Output schema for destination-mongodb

Each stream is output to its own MongoDB collection. Each collection contains three fields:

  • _id: an identifier assigned to each processed document. Its MongoDB field type is String.
  • _airbyte_emitted_at: a timestamp showing when the event was pulled from the data source. Its MongoDB field type is Timestamp.
  • _airbyte_data: a json blob containing the event data. Its MongoDB field type is Object.

Supported sync modes

Sync modeSupported?
Full Refresh - OverwriteYes
Full Refresh - AppendYes
Full Refresh - Overwrite + DedupedNo
Incremental Sync - AppendYes
Incremental Sync - Append + DedupedNo

Get started with Airbyte Cloud

Airbyte Cloud only supports TLS-encrypted connections to your MongoDB instance. Otherwise, you can follow the open-source instructions below.

Get started with Airbyte Open Source

Requirements

To use the MongoDB destination, you need:

  • A MongoDB server

Permissions

You need a MongoDB user who can create collections and write documents. Creating a dedicated Airbyte user is strongly recommended.

Target database

Choose an existing database or create one to store data synced from Airbyte.

Set up the MongoDB destination

You can now configure MongoDB as a destination in the UI. You need the following information:

  • Standalone MongoDb instance
    • Host: URL of the database
    • Port: Port used to connect to the database
    • TLS: indicates whether to create an encrypted connection
  • Replica Set
    • Server addresses: the members of a replica set
    • Replica Set: A replica set name
  • MongoDb Atlas Cluster
    • Cluster URL: URL of a cluster to connect to
  • Database
  • Username
  • Password

For more information about the configuration parameters, see MongoDb Documentation.

Connect through an SSH tunnel

Airbyte can connect to a MongoDB instance through an SSH Tunnel. You might need this connection when direct access is unavailable or conflicts with security policy. For example, the instance may lack a public IP address.

With an SSH tunnel, you configure Airbyte to connect through an intermediate server, also called a bastion server. This server has direct access to MongoDB. Airbyte connects to the bastion, which then connects directly to the server.

This feature requires additional configuration when you create the source. The following steps explain each configuration setting.

  1. Configure every source field as usual, except SSH Tunnel Method.
  2. SSH Tunnel Method defaults to No Tunnel, which means a direct connection. To use an SSH Tunnel, choose SSH Key Authentication or Password Authentication.
    1. Choose Key Authentication when using an RSA private key to establish the SSH Tunnel. See below for more information about generating this key.
    2. Choose Password Authentication when using a password to establish the SSH Tunnel.
  3. SSH Tunnel Jump Server Host identifies the intermediate bastion server that Airbyte connects to. Enter a hostname or an IP Address.
  4. SSH Connection Port is the bastion server port used for the SSH connection. The default SSH port is 22. Use this default unless you’ve explicitly changed it.
  5. SSH Login Username is the username Airbyte uses to connect to the bastion server. This is NOT the TiDB username.
  6. If you use Password Authentication, set SSH Login Username to the previous User’s password. If you use SSH Key Authentication, use the OS-user’s password, not the TiDB password. Airbyte uses this OS-user to run commands on the bastion.
  7. If you use SSH Key Authentication, set SSH Private Key to the RSA Private Key used for the SSH connection. Include the full key file contents, starting with -----BEGIN RSA PRIVATE KEY----- and ending with -----END RSA PRIVATE KEY-----.

Naming conventions

The following information comes from the MongoDB Limits and Thresholds documentation.

Database name case sensitivity

MongoDB database names are case-insensitive, so they can’t differ only by character case.

Database name restrictions on Windows

For MongoDB deployments on Windows, database names can’t contain these characters: /. ”$<>:|?

Database names also can’t contain the null character.

Database name restrictions on Unix and Linux

For MongoDB deployments on Unix and Linux systems, database names can’t contain these characters: /. ”$

Database names also can’t contain the null character.

Database name length

Database names can’t be empty and must contain fewer than 64 characters.

Collection name restrictions

Collection names should start with an underscore or letter. They can’t:

  • contain the $.
  • be an empty string (e.g. "").
  • contain the null character.
  • begin with the system. prefix. (Reserved for internal use.)

Namespace support

This destination supports namespaces. The namespace maps to a MongoDB database.

Changelog

Expand to review
VersionDatePull RequestSubject
0.2.02023-06-2727781License Update: Elv2
0.1.92022-11-0818892Adds check for TLS flag
0.1.82022-10-2618280Adds SSH tunneling
0.1.72022-09-0216025Remove additionalProperties:false from spec
0.1.62022-08-0215211Fix standard mode
0.1.52022-07-2714561Change Airbyte Id from MD5 to SHA256
0.1.42022-02-1410256(unpublished) Add -XX:+ExitOnOutOfMemoryError JVM option
0.1.32021-12-308809Update connector fields title/description
0.1.22021-10-186945Create a secure-only MongoDb destination
0.1.12021-09-296536Destination MongoDb: added support via TLS/SSL
Advanced Analytica

Most businesses are sitting on information they can’t use at speed. The people who hold it are busy, and when they leave, the knowledge goes with them. The documents are static, and the processes around them weren’t designed for AI.

Advanced Analytica works with these businesses to turn that information into structured data assets. Once they're integrated, AI systems can govern it, and the business can use it. We bring in the strategy, the specification and the AI skills to rework how the business operates.

MOVE FAST. STAY SAFE.