30 lines
797 B
Markdown
30 lines
797 B
Markdown
# 04-ingestion: Airbyte Data Ingestion
|
|
|
|
Airbyte OSS for data ingestion and ETL (standalone deployment).
|
|
|
|
## Services
|
|
|
|
- **airbyte**: All-in-one Airbyte container (webapp + server + worker + temporal)
|
|
|
|
## Run
|
|
|
|
```bash
|
|
docker compose --env-file ../.env.global up -d
|
|
```
|
|
|
|
## Access
|
|
|
|
- Web UI: http://localhost:8000
|
|
- Configure in Nginx to route domain to `airbyte:8000`
|
|
|
|
## Note
|
|
|
|
This uses Airbyte standalone image for simplicity. For production with high workload:
|
|
- Consider using full Airbyte stack with separate containers
|
|
- Or use `abctl` (Airbyte Command Line Tool)
|
|
- See: https://docs.airbyte.com/deploying-airbyte/docker-compose
|
|
|
|
## First Time Setup
|
|
1. Create database: `docker exec postgres psql -U postgres -c "CREATE DATABASE airbyte;"`
|
|
2. Access webapp and configure sources/destinations
|