SelectDB
The destination-selectdb destination is based on SelectDB copy into . It supports batch rollback and uses http/https put request.
Sync overview
Output schema
Each stream is written to its own SelectDB table. Each table contains three columns:
_airbyte_ab_id: a uuid that Airbyte assigns to each processed event. The column type in Doris isVARCHAR(40)._airbyte_emitted_at: a timestamp showing when the event was pulled from the data source. The column type in Doris isBIGINT._airbyte_data: a json blob containing the event data. The column type in SelectDB isString.
Features
This section should contain a table in the following format:
| Feature | Supported?(Yes/No) | Notes |
|---|---|---|
| Full Refresh Sync | Yes | |
| Incremental - Append Sync | Yes | |
| Incremental - Append + Deduped | No | |
| For databases, WAL/Logical replication | Yes |
Performance considerations
Writes are processed in batches. mini records may affect performance.
Importing multiple tables generates multiple transactions. Split these transactions as much as possible.
Getting started
Requirements
To use the SelectDB destination, you’ll need:
- A SelectDB server and your
- Ensure Airbyte can access your SelectDB http port and mysql query port.
- Ensure Airbyte can access your SelectDB host.
- If you use a public network, add your airbyte public network IP to your SelectDB ip whitelist.
- Ensure your SelectDB user has read/write permissions for the relevant tables.
Target database and tables
Choose a database to store data synced from Airbyte.
Prepare this database before syncing. Match the table’s column names and their order as closely as possible.
Set up access parameters
- loadURL
- jdbcURL
- clustername
- username
- password
- database
Changelog
Expand to review
| Version | Date | Pull Request | Subject |
|---|---|---|---|
| 0.1.0 | 2023-04-03 | #20881 | Initial release SelectDB Destination |