sriphat-dataplatform/04-ingestion/README.md

04-ingestion: Airbyte Data Ingestion

Airbyte OSS for data ingestion and ETL (multi-container deployment).

Services

• airbyte-proxy: Public entrypoint (UI/API gateway)
• server: Airbyte backend
• worker: Runs sync jobs and launches connector containers
• webapp: Airbyte UI
• airbyte-temporal: Workflow engine

Run

docker compose --env-file ../.env.global up -d

Access

• Web UI: http://localhost:8000
• Configure in Nginx to route domain to airbyte-proxy:8000

Note

This deployment pins Airbyte images to avoid :latest tag issues.

First Time Setup

1. Create database: docker exec postgres psql -U postgres -c "CREATE DATABASE airbyte;"
2. Access webapp and configure sources/destinations