Local JSON
:::danger
Use this destination on a local workstation. It won’t work on Kubernetes.
:::
Overview
This destination writes data to the local filesystem of the host running Airbyte. Data is written to /tmp/airbyte_local by default. To change this location, modify Airbyte’s LOCAL_ROOT environment variable.
Sync overview
Output schema
Each stream is written to its own file. Each file contains a collection of json objects with three fields:
_airbyte_ab_id: a uuid that Airbyte assigns to each processed event._airbyte_emitted_at: a timestamp showing when the event was pulled from the data source._airbyte_data: a json blob containing the extracted data.
Performance
Your filesystem’s write speed limits this integration’s performance.
Supported sync modes
| Sync mode | Supported? |
|---|---|
| Full Refresh - Overwrite | Yes |
| Full Refresh - Append | Yes |
| Full Refresh - Overwrite + Deduped | No |
| Incremental Sync - Append | Yes |
| Incremental Sync - Append + Deduped | No |
Get started
The destination_path always starts with /local, whether you specify it or not. Any directory nesting within local maps to the local mount.
By default, the LOCAL_ROOT env variable in the .env file is set to /tmp/airbyte_local.
Docker mounts the local mount at LOCAL_ROOT. Therefore, /local is substituted with /tmp/airbyte_local by default.
:::caution
Make sure Docker Desktop can access /tmp and /private on MacOS. The /tmp symlink points to /private, so access won’t work otherwise. Allow access through “File sharing” in Settings -> Resources -> File sharing -> add the one or two above folder, then select the “Apply & restart” button.
:::
Example
- Set
destination_pathto/local/cars/models. - Use the
/tmp/airbyte_localdefault for the local mount. - All data is then written to the
/tmp/airbyte_local/cars/modelsdirectory.
Access replicated files
If Airbyte runs on your computer, open file:///tmp/airbyte_local in your browser to view the replicated data locally. If that fails, or Airbyte runs remotely, follow these steps to access the replicated files:
- Access the scheduler container using
docker exec -it airbyte-server bash. - Go to the default local mount using
cd /tmp/airbyte_local. - Go to the replicated file directory specified when you created the destination, using
cd /{destination_path}. - List files containing the replicated data using
ls. - Run
cat {filename}to display the data in a specific file.
You can also copy the output file to your host machine. This command copies it to your current working directory:
docker cp airbyte-server:/tmp/airbyte_local/{destination_path}/{filename}.jsonl .
Note: If you run Airbyte on Windows using Docker backed by WSL2, follow similar steps or see link for another approach.
Namespace support
This destination doesn’t support namespaces.