Advanced Analytica Advanced Analytica: IBOM
← All connectors
TiDB logo

Source connector

TiDB

Set up TiDB as a source and move its data into your chosen destination.

Version 0.3.5 community alpha

TiDB

Overview

TiDB (/’taɪdiːbi:/, where “Ti” stands for Titanium) is an open-source, distributed NewSQL database for Hybrid Transactional and Analytical Processing (HTAP) workloads. It is MySQL compatible, with horizontal scalability, strong consistency and high availability. You can deploy TiDB on-premise or in-cloud.

The TiDB source supports Full Refresh and Incremental syncs. You can copy only new or updated data. Alternatively, copy every row from the tables and columns you’ve set up whenever a sync runs.

Resulting schema

The TiDB source preserves your database schema. However, the connected destination may alter it. See the destination’s documentation for details.

Features

FeatureSupportedNotes
Full Refresh SyncYes
Incremental - Append SyncYes
Replicate Incremental DeletesYes
Change Data CaptureNo
SSL SupportYes
SSH Tunnel ConnectionYes

Get started

Requirements

  1. TiDB v4.0 or above
  2. Allow Airbyte to connect to your TiDB database if they’re in separate VPCs
  3. (Optional) Create a dedicated, read-only Airbyte user with access to every table required for replication

Note: When you connect to TiDB Cloud with TLS enabled, specify a TLS protocol in the JDBC parameters. For example, use enabledTLSProtocols=TLSv1.2 or enabledTLSProtocols=TLSv1.3.

Setup guide

1. Make your database accessible to Airbyte

Access depends on your network setup. To confirm Airbyte can connect to your TiDB instance, use the check connection tool in the UI.

This optional step gives you better permission control and auditing. Alternatively, you can connect Airbyte using an existing database user.

Run the following commands against your database to create a dedicated user:

CREATE USER 'airbyte'@'%' IDENTIFIED BY 'your_password_here';

Then grant the user access to the relevant database:

GRANT SELECT ON <database name>.* TO 'airbyte'@'%';

3. Complete the setup

Your database user is now ready to use with Airbyte.

Connect through an SSH tunnel

Airbyte can connect to a TiDB instance through an SSH tunnel. Use this option when direct database access is impossible or conflicts with your security policy. For example, the database may not have a public IP address.

An SSH tunnel connects Airbyte through an intermediate server, also called a bastion server. This server must have direct database access. Airbyte connects to the bastion server, which then connects directly to the database server.

This feature requires extra configuration when you create the source. The following steps explain each configuration field.

  1. Configure every source field as usual, except SSH Tunnel Method.
  2. SSH Tunnel Method defaults to No Tunnel, which means a direct connection. For an SSH tunnel, choose SSH Key Authentication or Password Authentication.
    1. Choose Key Authentication to establish the SSH tunnel using an RSA private key. See below for more information about generating this key.
    2. Choose Password Authentication to establish the SSH tunnel using a password.
  3. SSH Tunnel Jump Server Host identifies the intermediate bastion server that Airbyte connects to. Enter its hostname or IP Address.
  4. SSH Connection Port is the bastion server port used for the SSH connection. The default SSH port is 22. Keep this default unless you’ve explicitly changed the port.
  5. SSH Login Username is the username Airbyte uses to connect to the bastion server. It isn’t the TiDB username.
  6. If you use Password Authentication, set SSH Login Username to the previous step’s User password. If you use SSH Key Authentication, enter the OS user’s password, not the TiDB password. Airbyte uses this OS user to run commands on the bastion.
  7. If you use SSH Key Authentication, set SSH Private Key to the RSA Private Key used for the SSH connection. Include the key file’s full contents, starting with -----BEGIN RSA PRIVATE KEY----- and ending with -----END RSA PRIVATE KEY-----.

Data type mapping

When synchronising data, TiDB data types map to the following data types:

TiDB TypeResulting TypeNotes
bit(1)boolean
bit(>1)base64 binary string
booleanboolean
tinyint(1)boolean
tinyintnumber
smallintnumber
mediumintnumber
intnumber
bigintnumber
floatnumber
doublenumber
decimalnumber
binarybase64 binary string
blobbase64 binary string
datestringISO 8601 date string. ZERO-DATE value will be converted to NULL. If column is mandatory, convert to EPOCH.
datetime, timestampstringISO 8601 datetime string. ZERO-DATE value will be converted to NULL. If column is mandatory, convert to EPOCH.
timestringISO 8601 time string. Values are in range between 00:00:00 and 23:59:59.
yearyear stringDoc
char, varchar with non-binary charsetstring
char, varchar with binary charsetbase64 binary string
tinyblobbase64 binary string
blobbase64 binary string
mediumblobbase64 binary string
longblobbase64 binary string
binarybase64 binary string
varbinarybase64 binary string
tinytextstring
textstring
mediumtextstring
longtextstring
jsonserialized json stringE.g. {"a": 10, "b": 15}
enumstring
setstringE.g. blue,green,yellow

Note: arrays support every type listed above, including custom types. However, the destination may de-nest them.

External resources

After setting up the TiDB source connector, you can read the following TiDB tutorial:

IP allow list

If you use Airbyte Cloud and your organisation restricts access by IP, add the Airbyte Cloud IP addresses to your allow list.

Changelog

Expand to review
VersionDatePull RequestSubject
0.3.52025-07-1062927Convert to new gradle build flow
0.3.42025-01-1051499Use a non root base image
0.3.32024-12-1849896Use a base image: airbyte/java-connector-base:1.0.0
0.3.22024-02-1335218Adopt CDK 0.20.4
0.3.12024-01-2434453bump CDK version
0.3.02023-12-1833485Remove LEGACY state
0.2.52023-06-2027212Fix silent exception swallowing in StreamingJdbcDatabase
0.2.42023-03-2220760Removed redundant date-time datatypes formatting
0.2.32023-03-0623455For network isolation, source connector accepts a list of hosts it is allowed to connect to
0.2.22022-12-1420346Consolidate date/time values mapping for JDBC sources
2022-10-1315535Update incremental query to avoid data missing when new data is inserted at the same time as a sync starts under non-CDC incremental mode
0.2.12022-09-0116238Emit state messages more frequently
0.2.02022-07-2614362Integral columns are now discovered as int64 fields.
0.1.52022-07-2514996Removed additionalProperties:false from spec
0.1.42022-07-2214714Clarified error message when invalid cursor column selected
0.1.32022-07-0414243Update JDBC string builder
0.1.22022-06-1713864Updated stacktrace format for any trace message errors
0.1.12022-04-2912480Query tables with adaptive fetch size to optimize JDBC memory consumption
0.1.02022-04-1911283Initial Release
Advanced Analytica

Most businesses are sitting on information they can’t use at speed. The people who hold it are busy, and when they leave, the knowledge goes with them. The documents are static, and the processes around them weren’t designed for AI.

Advanced Analytica works with these businesses to turn that information into structured data assets. Once they're integrated, AI systems can govern it, and the business can use it. We bring in the strategy, the specification and the AI skills to rework how the business operates.

MOVE FAST. STAY SAFE.