Advanced Analytica Advanced Analytica: IBOM
← All connectors
Amazon SQS logo

Source connector

Amazon SQS

Set up Amazon SQS as a source and move its data into your chosen destination.

Version 1.0.15 community alpha

Amazon SQS

Overview

The Amazon SQS source syncs the SQS API. See https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/Welcome.html. It supports full refresh sync. You can view it in builder with low-code support.

Get started

Requirements

  • AWS IAM Access Key
  • AWS IAM Secret Key
  • AWS SQS Queue URL
  • AWS Region
  • Action target

Setup guide

Supported streams

This source can sync the following core Actions as streams:

Other Actions are in beta and might require more or fewer parameters.

Features

FeatureSupported?(Yes/No)Notes
Full Refresh SyncYes
Incremental SyncNo
NamespacesNo

Performance

Lookup requests for the RecieveMessage stream are limited to two per second, per account and per region. The connector retries after throttling errors. However, the sync will fail if errors continue after multiple retries. This can happen when you set up many connector instances using the same account and region.

Output schema

This source outputs one stream for the configured SQS Queue. Each stream record contains three fields:

  • id (a UUIDv4 as a STRING)
  • body (message body as a STRING)
  • attributes (attributes of the messages as an OBJECT or NULL)

Properties

The required properties are ‘Queue URL’, ‘AWS Region’ and ‘Delete Messages After Read’, as shown in bold below.

  • Queue URL (STRING)
    • The queue’s full AWS endpoint URL, e.g. https://sqs.eu-west-1.amazonaws.com/1234567890/example-queue-url
  • AWS Region (STRING)
    • The region code for the SQS Queue, e.g. eu-west-1
  • Max Batch Size (INTEGER)
    • The maximum number of messages to consume in one poll, e.g. 5
    • Minimum of 1, maximum of 10
    • Default: 10
  • Max Wait Time (INTEGER)
    • The maximum polling time, in seconds, before committing or timing out a batch
    • Polling ends earlier if the batch reaches Max Batch Size
    • Minimum of 1, maximum of 20
    • Default: 20
  • Message Attributes To Return (STRING)
    • A comma-separated list of Attributes to return for each message
    • Default: All
  • Message Visibility Timeout (INTEGER)
    • The time, in seconds, that a read message stays hidden from other consumers
    • After this timeout, the message isn’t deleted and can be read again
    • Default: 20
  • AWS IAM Access Key ID (STRING)
    • The Access Key for the IAM User with permissions on this Queue
  • AWS IAM Secret Key (STRING)
    • The Secret Key for the IAM User with permissions on this Queue
  • Target (STRING)
    • The targeted action resource for the fetch

IP allow list

If you use Airbyte Cloud and restrict access to specific IPs, add Airbyte Cloud IP addresses to your allow list.

Changelog

Expand to review
VersionDatePull RequestSubject
1.0.152025-05-1059775Update dependencies
1.0.142025-05-0359344Update dependencies
1.0.132025-04-2658684Update dependencies
1.0.122025-04-1958293Update dependencies
1.0.112025-04-1257609Update dependencies
1.0.102025-04-0557146Update dependencies
1.0.92025-03-2956614Update dependencies
1.0.82025-03-2256083Update dependencies
1.0.72025-03-0854832Update dependencies
1.0.62025-03-0154738fix: Update source-amazon-sqs to use nltk 3.9.1 or higher
1.0.52025-02-2254271Update dependencies
1.0.42025-02-1553936Update dependencies
1.0.32025-02-0152878Update dependencies
1.0.22025-01-2552163Update dependencies
1.0.12025-01-1851742Update dependencies
1.0.02024-11-0741064Migrate to low code
0.1.12024-01-03#33924Add new ap-southeast-3 AWS region
0.1.02021-10-10#0000Initial version
Advanced Analytica

Most businesses are sitting on information they can’t use at speed. The people who hold it are busy, and when they leave, the knowledge goes with them. The documents are static, and the processes around them weren’t designed for AI.

Advanced Analytica works with these businesses to turn that information into structured data assets. Once they're integrated, AI systems can govern it, and the business can use it. We bring in the strategy, the specification and the AI skills to rework how the business operates.

MOVE FAST. STAY SAFE.