sriphat-dataplatform/DEPLOYMENT.md

# Sriphat Data Platform - Deployment Guide

## 📋 Architecture Overview

```
┌─────────────────────────────────────────────────────────────┐
│                    Nginx Proxy Manager                       │
│              (Gateway + SSL + Domain Routing)                │
└─────────────────────────────────────────────────────────────┘
                              │
        ┌─────────────────────┼─────────────────────┐
        │                     │                     │
┌───────▼────────┐   ┌────────▼────────┐   ┌───────▼────────┐
│   Keycloak     │   │   API Service   │   │    Superset    │
│     (SSO)      │   │   (FastAPI)     │   │      (BI)      │
└────────────────┘   └─────────────────┘   └────────────────┘
        │                     │                     │
        └─────────────────────┼─────────────────────┘
                              │
                    ┌─────────▼─────────┐
                    │    PostgreSQL     │
                    │  (Data Warehouse) │
                    └───────────────────┘
                              │
                    ┌─────────▼─────────┐
                    │     Airbyte       │
                    │  (Data Ingestion) │
                    └───────────────────┘
```

## 🚀 Quick Start

### Prerequisites
- Docker & Docker Compose installed
- Minimum 8GB RAM
- 50GB disk space

### Step 1: Clone & Configure
```bash
cd e:\git3\sriphat-dataplatform

# Review and update credentials in .env.global
notepad .env.global
```

### Step 2: Start All Services
```bash
# On Linux/Mac
bash start-all.sh

# On Windows (PowerShell)
bash start-all.sh
# OR manually:
# 1. cd 00-network && bash create-network.sh
# 2. cd ../01-infra && docker compose --env-file ../.env.global up -d
# 3. Wait 30 seconds for PostgreSQL
# 4. cd ../03-apiservice && docker compose --env-file ../.env.global up --build -d
# 5. cd ../04-ingestion && docker compose --env-file ../.env.global up -d
# 6. cd ../06-analytics && docker compose --env-file ../.env.global up -d
```

### Step 3: Verify Services
```bash
docker ps
```

You should see:
- nginx-proxy-manager
- keycloak
- postgres
- apiservice
- airbyte-webapp, airbyte-server, airbyte-worker, airbyte-temporal
- superset

## 🔑 Access Points

| Service | URL | Default Credentials |
|---------|-----|---------------------|
| **Nginx Proxy Manager** | http://localhost:8021 | admin@example.com / changeme |
| **Keycloak Admin** | http://localhost:8080 | See KEYCLOAK_ADMIN in .env.global |
| **Airbyte** | http://localhost:8000 | No auth (setup via Nginx) |
| **API Service** | Configure via Nginx | See ADMIN_USERNAME in .env.global |
| **Superset** | Configure via Nginx | See SUPERSET_ADMIN_USERNAME in .env.global |

## 📝 Post-Installation Setup

### 1. Configure Nginx Proxy Manager

1. Access http://localhost:8021
2. Login with default credentials (change on first login)
3. Add Proxy Hosts:

**API Service:**
- Domain: `api.sriphat.local` (or your domain)
- Forward Hostname: `apiservice`
- Forward Port: `8000`
- Custom locations:
  - Location: `/apiservice`
  - Forward Hostname: `apiservice`
  - Forward Port: `8000`

**Keycloak:**
- Domain: `auth.sriphat.local`
- Forward Hostname: `keycloak`
- Forward Port: `8080`

**Superset:**
- Domain: `bi.sriphat.local`
- Forward Hostname: `superset`
- Forward Port: `8088`

**Airbyte:**
- Domain: `etl.sriphat.local`
- Forward Hostname: `airbyte`
- Forward Port: `8000`

### 2. Setup Keycloak SSO

1. Access Keycloak admin console
2. Create new Realm: `sriphat`
3. Create Clients:
   - **superset-client** (for Superset OAuth)
   - **apiservice-client** (for API Service)
4. Configure OIDC settings
5. Create Users and assign roles

### 3. Initialize API Service

```bash
# Access admin UI
# http://api.sriphat.local/apiservice/admin/

# Create API Client
# 1. Go to ApiClient menu
# 2. Create new client (e.g., "mobile-app")

# Generate API Key
curl -X POST "http://api.sriphat.local/apiservice/admin/api-keys/generate?client_id=1&permissions=feed.checkpoint:write&name=production-key" \
  -H "Cookie: session=<your-admin-session>"

# Test API
curl -X POST "http://api.sriphat.local/apiservice/api/v1/feed/checkpoint" \
  -H "Authorization: Bearer <api-key>" \
  -H "Content-Type: application/json" \
  -d '[{"id":1,"hn":123,"vn":456,"location":"OPD","type":"Scan","timestamp_in":"2026-02-16T10:00:00","timestamp_out":null,"waiting_time":null,"bu":"SRIPHAT"}]'
```

### 4. Configure Airbyte Sources

1. Access Airbyte UI
2. Setup Sources:
   - SQL Server (HIS Database)
   - Oracle (Lab System)
   - REST API endpoints
3. Setup Destination:
   - PostgreSQL (host: `postgres`, database: `postgres`, schemas: `raw_data`)
4. Create Connections and schedule syncs

### 5. Setup Superset Dashboards

1. Access Superset
2. Add Database Connection:
   - PostgreSQL: `postgresql://postgres:password@postgres:5432/postgres`
3. Create Datasets from `analytics` schema
4. Build Dashboards

## 🔒 Security Checklist

- [ ] Change all default passwords in `.env.global`
- [ ] Enable SSL in Nginx Proxy Manager (Let's Encrypt)
- [ ] Configure Keycloak with hospital LDAP/AD
- [ ] Enable Row-Level Security (RLS) in PostgreSQL
- [ ] Restrict network access (firewall rules)
- [ ] Setup backup strategy for PostgreSQL data
- [ ] Enable audit logging in all services
- [ ] Configure session timeouts

## 🛠️ Maintenance

### View Logs
```bash
# All services
docker compose -f 01-infra/docker-compose.yml logs -f

# Specific service
docker logs -f apiservice
docker logs -f keycloak
docker logs -f superset
```

### Backup Database
```bash
docker exec postgres pg_dump -U postgres postgres > backup_$(date +%Y%m%d).sql
```

### Restore Database
```bash
docker exec -i postgres psql -U postgres postgres < backup_20260216.sql
```

### Update Services
```bash
# Stop all
bash stop-all.sh

# Pull latest images
docker compose -f 01-infra/docker-compose.yml pull
docker compose -f 04-ingestion/docker-compose.yml pull
docker compose -f 06-analytics/docker-compose.yml pull

# Rebuild API service
cd 03-apiservice
docker compose --env-file ../.env.global build

# Start all
cd ..
bash start-all.sh
```

## 🐛 Troubleshooting

### PostgreSQL connection issues
```bash
# Check if PostgreSQL is ready
docker exec postgres pg_isready -U postgres

# Check schemas
docker exec postgres psql -U postgres -c "\dn"
```

### Keycloak not starting
```bash
# Check logs
docker logs keycloak

# Ensure PostgreSQL is ready first
docker restart keycloak
```

### API Service can't connect to DB
```bash
# Verify network
docker network inspect shared_data_network

# Check environment variables
docker exec apiservice env | grep DB_
```

### Airbyte worker issues
```bash
# Ensure Docker socket is mounted
docker exec airbyte-worker ls -la /var/run/docker.sock

# Check Temporal
docker logs airbyte-temporal
```

## 📊 Monitoring

### Resource Usage
```bash
docker stats
```

### Health Checks
```bash
# PostgreSQL
curl http://localhost:5432 || echo "PostgreSQL internal only - OK"

# Nginx Proxy Manager
curl -I http://localhost:81

# Keycloak
curl -I http://localhost:8080

# API Service (via network)
docker exec nginx-proxy-manager curl -I http://apiservice:8000/apiservice/docs
```

## 🔄 Scaling

### Increase API Service Workers
Edit `03-apiservice/Dockerfile`:
```dockerfile
CMD ["gunicorn","-k","uvicorn.workers.UvicornWorker","app.main:app","--bind","0.0.0.0:8000","--workers","4"]
```

### Add Read Replicas (PostgreSQL)
- Configure streaming replication
- Update connection strings for read-only queries

## 📞 Support

For issues:
1. Check logs: `docker logs <container-name>`
2. Verify network: `docker network inspect shared_data_network`
3. Review configuration: `.env.global`
4. Restart specific service: `docker restart <container-name>`