description: >- Keen is a fully managed platform for event streaming and analytics.
Keen
Overview
The Airbyte Keen destination streams data from any Airbyte Source into Keen for storage, analysis and visualisation. Keen is a flexible, fully managed platform for event streaming and analytics. It enables anyone to deliver custom, embeddable dashboards in minutes, not months.
Prerequisites
- If you use Airbyte Open Source with the Postgres source connector, upgrade your Airbyte platform to version
v0.40.0-alphaor newer. You must also upgrade your Keen connector to version0.2.4or newer.
Sync overview
Output schema
Each replicated Airbyte stream outputs data to a corresponding event collection in Keen. Event collections store data in JSON format. Each collection inherits the stream name. The connector removes all non-alphanumeric characters, except .’, ‘-’, ‘_’, and whitespace characters. Where possible, the connector infers the record’s timestamp value and uses it to override the special keen.timestamp field.
Features
| Feature | Supported?(Yes/No) | Notes |
|---|---|---|
| Full Refresh Sync | Yes | |
| Incremental - Append Sync | Yes | |
| Incremental - Append + Deduped | No | |
| Namespaces | No |
Get started
Requirements
Before using the Keen destination, create a Keen account if you don’t already have one.
Once you have a Keen account, use these credentials to set up the connector:
- A Keen Project ID
- A Keen Master API key associated with the project
See the setup guide for more information about getting started.
Setup guide
Keen Project
If you haven’t created a project for streaming your data:
Sign in to the Keen application and add a new project. Select the ‘Add New’ link beside the Projects label in the left-hand side panel. Then name the project.
A project works as a data silo. Its data remains completely separate from data in other projects. Create separate projects for each application and for Dev and Prod environments.
Open the ‘Access’ section and copy your Project ID and Master API Key.
If you’ve already created a project:
Open the ‘Access’ tab and copy your Project ID and Master API Key.
API Key and Project ID
The Keen Connector uses the Keen Kafka Inbound Cluster to stream data. Authentication requires your Project ID and Master Key. To find them, open the Access tab from the left-hand side panel. Then check the Project Details section.
Important: This destination requires the Project’s Master Key.
Timestamp inference
The Infer Timestamp field controls whether the connector infers the keen.timestamp field from event data recorded in the source application. This feature supports historical data synchronisation, giving you the full power of Keen’s time series analytics. By default, this property is set to true. If you turn it off, keen.timestamp uses the date and time when Keen recorded the data.
Set up the Keen destination in Airbyte
You now have all the parameters required to configure the Keen destination.
- Project ID
- Master API Key
- Infer Timestamp
Connect your first source, then open the Keen application. You can run custom analysis on your data and build interactive dashboards for key stakeholders.
If you have questions, contact [email protected] for help.
Changelog
Expand to review
| Version | Date | Pull Request | Subject |
|---|---|---|---|
| 0.2.4 | 2022-08-04 | 15291 | Update Keen destination to use outputRecordCollector to properly store state |
| 0.2.3 | 2022-06-17 | 13864 | Updated stacktrace format for any trace message errors |
| 0.2.1 | 2021-12-30 | 8809 | Update connector fields title/description |
| 0.2.0 | 2021-09-10 | 5973 | Fix timestamp inference for complex schemas |
| 0.1.0 | 2021-08-18 | 5339 | Keen Destination Release! |