Advanced Analytica Advanced Analytica: IBOM
← All connectors
Apache Doris logo

Destination connector

Apache Doris

Send governed data into Apache Doris with a clear, practical setup guide.

Version 0.1.0 archived alpha

Doris

destination-doris is built on Apache Doris stream load. It supports batch rollback and sends HTTP/HTTPS PUT requests.

Sync overview

Output schema

Each stream is written to its own table in Doris. Each table contains three columns:

  • _airbyte_ab_id: a UUID Airbyte assigns to each processed event. In Doris, the column type is VARCHAR(40).
  • _airbyte_emitted_at: a timestamp showing when the event was pulled from the data source. In Doris, the column type is BIGINT.
  • _airbyte_data: a JSON blob containing the event data. In Doris, the column type is String.

Features

Use the following table format:

FeatureSupported?(Yes/No)Notes
Full Refresh SyncYes
Incremental - Append SyncYes
Incremental - Append + DedupedNoit will soon be realized
For databases, WAL/Logical replicationYes

Performance considerations

Writes are processed in batches. Small record batches may affect performance. Importing multiple tables creates multiple Doris stream load transactions. Split imports as much as possible.

Getting started

Requirements

To use the Doris destination, you’ll need:

  • A Doris server running version 0.14 or above
  • Airbyte must be able to access your Doris FE HTTP port.
  • Airbyte must be able to access your Doris database host.
  • Your Doris user must have read and write permissions for specific tables.

Target database and tables

Choose a database to store data synced from Airbyte. Prepare the tables that will store this data. Match their column names and order as closely as possible.

Set up access parameters

  • Host
  • HttpPort
  • QueryPort
  • Username
  • Password
  • Database

Changelog

Expand to review
VersionDatePull RequestSubject
0.1.02022-11-1417884Initial Commit
Advanced Analytica

Most businesses are sitting on information they can’t use at speed. The people who hold it are busy, and when they leave, the knowledge goes with them. The documents are static, and the processes around them weren’t designed for AI.

Advanced Analytica works with these businesses to turn that information into structured data assets. Once they're integrated, AI systems can govern it, and the business can use it. We bring in the strategy, the specification and the AI skills to rework how the business operates.

MOVE FAST. STAY SAFE.