Compare commits

..

8 Commits

Author SHA1 Message Date
jigoong
76398c3de6 feat(apiservice): add edit client/key functionality in API Management page
- PATCH /admin/api-keys/clients/{id} — update client name and is_active
- PATCH /admin/api-keys/{id} — update key name and permissions
- Edit Client modal with name field and active/inactive toggle
- Edit Key modal with name field and permissions JSON textarea (pre-filled)
- Fix JS syntax error: use data-* attributes instead of inline JSON in onclick
2026-06-09 00:41:36 +07:00
jigoong
3a5f9e9001 feat: replace SQLAdmin with Keycloak-protected API management page
- Disable SQLAdmin basic auth (comment out mount_admin, statics, redirect)
- Add /api-management page (Keycloak admin role required)
- Add admin_api_keys.py: REST endpoints for list/create clients and keys
- Add api_management.html: manage API clients, keys, permissions with copy-once key display
- Update index.html: API Management link -> /api-management
- Update auth middleware: add /api-management and /admin/users to PROTECTED_PATHS
- Add CHANGES-2026-06-04.md dev notes
2026-06-04 18:22:22 +07:00
jigoong
e4d32b86cb feat: add VOC data endpoint (POST /api/v1/voc-data)
- Add VocDataIn schema (date, topic, sub_topic, level, depart_id, dep_name)
- Add RawVocData SQLAlchemy model (rawdata.raw_voc_data, BIGSERIAL PK)
- Add POST /api/v1/voc-data endpoint with voc.data:write permission
- Dual-write to local PostgreSQL + Supabase
- Table auto-created on startup via Base.metadata.create_all()
2026-06-04 18:22:14 +07:00
jigoong
ee473aca8f fix: finance upload filepath bug and add extra_hosts for keycloak auth flow
- fix NameError: filepath undefined in trigger_airflow call (use filepath_stored)
- add extra_hosts ai.sriphat.com:192.168.100.8 for container DNS resolution
  (required for KEYCLOAK_SERVER_URL=http://ai.sriphat.com/keycloak/ to work
   inside Docker — host nginx on .8:80 routes /keycloak/ to Keycloak container)
2026-05-27 01:28:57 +07:00
jigoong
a587be08bd feat: MinIO integration — bucket finance, API service upload, Nginx routing
- 01-infra/nginx-configs: add MinIO /minio/ and /minio-console/ location blocks
  (port 9000 S3 API, port 9001 Console UI, path stripping via rewrite)
- 03-apiservice: integrate MinIO minio-python SDK for file upload
  - requirements.txt: add minio==7.2.11
  - app/core/config.py: add MINIO_ENDPOINT, ACCESS_KEY, SECRET_KEY, BUCKET_FINANCE, USE_SSL
  - app/services/minio_client.py: new — upload_file(), get_presigned_url(), delete_file()
  - app/routes/pages.py: replace local /data/uploads/ write with MinIO upload to finance bucket
  - docker-compose.yml: pass MinIO env vars to container
  - .env.example: document MinIO vars
- 07-minio/.env.example: add MINIO_SVC_ACCESS_KEY/SECRET_KEY section
- 07-minio/README.md: add Python minio SDK and Airflow DAG usage guide
- CLAUDE.md: project context (servers, SSH, paths, service distribution)
- document-obsidiant/: initial Obsidian docs for all services
2026-05-20 17:42:39 +07:00
jigoong
9dcf24eeb7 update config for limit resouce size 2026-05-08 22:18:32 +07:00
jigoong
1dba772e62 update configuration docker setup for data platform 2026-05-07 17:57:42 +07:00
jigoong
ce949dcc8f add supavisor config 2026-03-06 17:37:47 +07:00
98 changed files with 10876 additions and 969 deletions

View File

@@ -4,6 +4,7 @@ TZ=Asia/Bangkok
DB_HOST=postgres
DB_PORT=5432
DB_PORT_EXPOSE=5435
DB_USER=postgres
DB_PASSWORD=Secure_Hospital_Pass_2026
DB_NAME=postgres
@@ -13,6 +14,7 @@ POSTGRES_PASSWORD=Secure_Hospital_Pass_2026
KEYCLOAK_ADMIN=admin
KEYCLOAK_ADMIN_PASSWORD=admin_secret_pass_2026
KEYCLOAK_DB_NAME=keycloak
SUPERSET_SECRET_KEY=superset_random_secret_key_change_me_2026
SUPERSET_ADMIN_USERNAME=admin
@@ -29,3 +31,14 @@ AIRBYTE_PORT=8030
AIRBYTE_BASIC_AUTH_USERNAME=
AIRBYTE_BASIC_AUTH_PASSWORD=
AIRBYTE_BASIC_AUTH_PROXY_TIMEOUT=900
# Dozzle - Docker Log Viewer & Monitoring
DOZZLE_PORT=9999
DOZZLE_LEVEL=info
DOZZLE_BASE=/dozzle
DOZZLE_HOSTNAME=Sriphat Main Server
DOZZLE_AUTH_PROVIDER=none
DOZZLE_RESTART_POLICY=unless-stopped
# Remote agents: Airbyte and Airflow on 192.168.100.9
# Format: host:port,host:port (comma-separated)
DOZZLE_REMOTE_AGENT=192.168.100.9:7007

2
.gitignore vendored
View File

@@ -12,3 +12,5 @@ ruff_cache/
*/data/
01-infra/letsencrypt/
.windsurf/
_daily-log/
daily-log/

View File

@@ -1,51 +1,76 @@
x-common-configs: &common-config
extra_hosts:
- "dev.sriphat.com:192.168.100.9"
pull_policy: ${DOCKER_PULL_POLICY:-missing}
services:
# nginx-proxy:
# image: jc21/nginx-proxy-manager:latest
# container_name: nginx-proxy-manager
# ports:
# - "8020:80"
# - "8043:443"
# - "8021:81"
# volumes:
# - ./data:/data
# - ./letsencrypt:/etc/letsencrypt
# environment:
# - TZ=${TZ:-Asia/Bangkok}
# env_file:
# - ../.env
# networks:
# - shared_data_network
# restart: unless-stopped
nginx-proxy:
image: jc21/nginx-proxy-manager:latest
image: nginx:latest
container_name: nginx-proxy-manager
ports:
- "8020:80"
- "8043:443"
- "8021:81"
volumes:
- ./data:/data
- ./letsencrypt:/etc/letsencrypt
environment:
- TZ=${TZ:-Asia/Bangkok}
env_file:
- ../.env.global
volumes:
- ./nginx-configs/default-all.conf:/etc/nginx/conf.d/default.conf:ro
networks:
- shared_data_network
restart: unless-stopped
<<: *common-config
keycloak:
image: quay.io/keycloak/keycloak:23.0
container_name: keycloak
command: start-dev
#command: start-dev
command: start-dev --http-relative-path /keycloak
env_file:
- ../.env.global
- ../.env
environment:
KC_BOOTSTRAP_ADMIN_USERNAME: ${KEYCLOAK_ADMIN}
KC_BOOTSTRAP_ADMIN_PASSWORD: ${KEYCLOAK_ADMIN_PASSWORD}
KC_DB: postgres
KC_DB_URL: jdbc:postgresql://postgres:5432/${KEYCLOAK_DB_NAME}
KC_DB_URL: jdbc:postgresql://postgres:${DB_PORT:-5432}/${KEYCLOAK_DB_NAME}
KC_DB_USERNAME: ${DB_USER}
KC_DB_PASSWORD: ${DB_PASSWORD}
KC_HOSTNAME_STRICT: "false"
KC_HTTP_ENABLED: "true"
KC_PROXY: edge
# passthrough
KC_HTTP_RELATIVE_PATH: "/keycloak"
KC_HOSTNAME_PATH: "/keycloak"
KC_HOSTNAME_STRICT_HTTPS: "true"
ports:
- "8080:8080"
- "8085:8080"
networks:
- shared_data_network
restart: unless-stopped
depends_on:
- postgres
<<: *common-config
postgres:
image: postgres:15-alpine
container_name: postgres
env_file:
- ../.env.global
- ../.env
environment:
POSTGRES_PASSWORD: ${DB_PASSWORD}
POSTGRES_USER: ${DB_USER}
@@ -58,13 +83,57 @@ services:
- shared_data_network
restart: unless-stopped
ports:
- "0.0.0.0:5435:5432"
- "0.0.0.0:${DB_PORT_EXPOSE:-5435}:5432"
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${DB_USER}"]
interval: 10s
timeout: 5s
retries: 5
redis:
# Redis is limited to 7.2-bookworm due to licencing change
# https://redis.io/blog/redis-adopts-dual-source-available-licensing/
image: redis:7.2-bookworm
expose:
- 6379
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 30s
retries: 50
start_period: 30s
restart: always
networks:
- shared_data_network
dozzle:
image: amir20/dozzle:latest
container_name: dozzle
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./data/dozzle:/data
ports:
- "${DOZZLE_PORT:-9999}:8080"
environment:
DOZZLE_LEVEL: ${DOZZLE_LEVEL:-info}
DOZZLE_BASE: ${DOZZLE_BASE:-/dozzle}
DOZZLE_HOSTNAME: ${DOZZLE_HOSTNAME:-Sriphat Main Server}
DOZZLE_NO_ANALYTICS: "true"
DOZZLE_ENABLE_ACTIONS: "true"
DOZZLE_AUTH_PROVIDER: ${DOZZLE_AUTH_PROVIDER:-none}
DOZZLE_REMOTE_AGENT: ${DOZZLE_REMOTE_AGENT:-}
TZ: ${TZ:-Asia/Bangkok}
networks:
- shared_data_network
restart: ${DOZZLE_RESTART_POLICY:-unless-stopped}
healthcheck:
test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:8080/dozzle/healthcheck"]
interval: 30s
timeout: 10s
retries: 3
start_period: 10s
<<: *common-config
networks:
shared_data_network:
external: true

View File

@@ -0,0 +1,5 @@
-- Create Keycloak database
CREATE DATABASE keycloak;
-- Grant privileges to postgres user
GRANT ALL PRIVILEGES ON DATABASE keycloak TO postgres;

View File

@@ -0,0 +1,8 @@
-- Create databases for Airbyte OSS
-- These databases will be used by the Airbyte deployment in 04-ingestion
-- Main Airbyte database
CREATE DATABASE airflow_db;
-- Grant permissions to postgres user
GRANT ALL PRIVILEGES ON DATABASE airflow_db TO postgres;

View File

@@ -0,0 +1,149 @@
# Nginx Proxy Manager - Site Configurations
Nginx site configurations สำหรับ handle subpath routing ของทุก service ใน Sriphat Data Platform
## 📋 Services และ Subpaths
| Service | Subpath | Backend | Port |
|---------|---------|---------|------|
| API Service | `/apiservice` | apiservice:8000 | 8040 |
| Supabase Studio | `/supabase` | sdp-studio:3000 | - |
| Supabase Kong API | `/supabase-api` | sdp-kong:8000 | - |
| Keycloak | `/keycloak` | keycloak:8080 | 8085 |
| Superset | `/superset` | superset:8088 | 8088 |
| Airflow | `/airflow` | airflow-webserver:8080 | - |
| DBT Docs | `/dbt` | dbt-docs:8080 | - |
| Dozzle | `/dozzle` | dozzle:8080 | 9999 |
## 🚀 การใช้งาน
### วิธีที่ 1: ใช้ Nginx Proxy Manager UI (แนะนำ)
1. เข้า Nginx Proxy Manager: `http://your-server:8021`
2. Login (default: admin@example.com / changeme)
3. ไปที่ **Proxy Hosts****Add Proxy Host**
4. กรอกข้อมูล:
- **Domain Names**: `ai.sriphat.com` (หรือ domain ของคุณ)
- **Scheme**: `http`
- **Forward Hostname/IP**: ดูจากตารางด้านบน
- **Forward Port**: ดูจากตารางด้านบน
5. ไปที่ tab **Advanced**
6. Copy config จากไฟล์ที่เกี่ยวข้อง (เช่น `apiservice.conf`) ไปวางใน **Custom Nginx Configuration**
7. กด **Save**
### วิธีที่ 2: Import Config Files โดยตรง
**⚠️ วิธีนี้ต้องการ access ไปยัง Nginx Proxy Manager data directory**
```bash
# 1. Copy config files ไปยัง Nginx Proxy Manager
cd /path/to/01-infra
docker cp nginx-configs/. nginx-proxy-manager:/data/nginx/custom/
# 2. Restart Nginx Proxy Manager
docker restart nginx-proxy-manager
# 3. ตรวจสอบ logs
docker logs nginx-proxy-manager -f
```
## 📝 Config Files
### `apiservice.conf`
FastAPI service with Keycloak authentication
- Handles `/apiservice/*` paths
- Preserves session cookies
- WebSocket support
### `supabase-studio.conf`
Supabase Studio UI
- Handles `/supabase/*` paths
- Rewrites paths for Studio
### `supabase-kong.conf`
Supabase REST API (Kong Gateway)
- Handles `/supabase-api/*` paths
- API key authentication
### `keycloak.conf`
Keycloak SSO
- Handles `/keycloak/*` paths
- Preserves authentication headers
### `superset.conf`
Apache Superset BI
- Handles `/superset/*` paths
- Session management
### `airflow.conf`
Apache Airflow (if deployed)
- Handles `/airflow/*` paths
- WebServer UI
### `dbt.conf`
DBT Documentation (if deployed)
- Handles `/dbt/*` paths
- Static documentation
## 🔧 การปรับแต่ง
### เปลี่ยน Domain
แก้ไข `server_name` ในแต่ละ config file:
```nginx
server_name ai.sriphat.com; # เปลี่ยนเป็น domain ของคุณ
```
### เปลี่ยน Backend Host/Port
แก้ไข `proxy_pass` directive:
```nginx
proxy_pass http://apiservice:8000; # เปลี่ยนตาม service ของคุณ
```
### เพิ่ม SSL/HTTPS
ใช้ Nginx Proxy Manager UI:
1. ไปที่ Proxy Host ที่ต้องการ
2. ไปที่ tab **SSL**
3. เลือก **Request a new SSL Certificate**
4. เลือก **Force SSL**
## 🐛 Troubleshooting
### 502 Bad Gateway
- ตรวจสอบว่า backend service ทำงานอยู่: `docker ps`
- ตรวจสอบ network: `docker network inspect shared_data_network`
- ดู logs: `docker logs <service-name>`
### 404 Not Found
- ตรวจสอบ path rewriting ใน config
- ดู nginx logs: `docker logs nginx-proxy-manager`
### Session/Cookie Issues
- ตรวจสอบ `proxy_cookie_path` directive
- ตรวจสอบ `X-Forwarded-*` headers
### WebSocket Connection Failed
- ตรวจสอบว่ามี WebSocket headers:
```nginx
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
```
## 📚 เอกสารเพิ่มเติม
- [Nginx Proxy Manager Documentation](https://nginxproxymanager.com/guide/)
- [Nginx Reverse Proxy Guide](https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/)
- [Nginx Subpath Configuration](https://www.nginx.com/blog/creating-nginx-rewrite-rules/)
## 🔐 Security Notes
1. **ใช้ HTTPS ใน Production** - Request SSL certificate ผ่าน Nginx Proxy Manager
2. **ตั้งค่า Access Lists** - จำกัดการเข้าถึงบาง services (เช่น Keycloak Admin)
3. **Enable Rate Limiting** - ป้องกัน DDoS attacks
4. **Update Regularly** - อัพเดท Nginx Proxy Manager เป็นประจำ
## 📞 Support
หากมีปัญหาหรือข้อสงสัย:
1. ตรวจสอบ logs: `docker logs nginx-proxy-manager -f`
2. ดู Nginx Proxy Manager UI → **Logs**
3. ตรวจสอบ backend service logs

View File

@@ -0,0 +1,67 @@
# Apache Airflow - Workflow Orchestration
# Subpath: /airflow
# Backend: airflow-webserver:8080
location /airflow {
# Remove /airflow prefix before forwarding
rewrite ^/airflow(/.*)$ $1 break;
# Forward to Airflow Webserver
proxy_pass http://airflow-webserver:8080;
# Preserve headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
# Important for Airflow subpath
proxy_set_header X-Script-Name /airflow;
# Session cookie handling
proxy_cookie_path / /airflow/;
# WebSocket support for real-time logs
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Timeouts (DAG runs can take time)
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
# Buffer settings
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
}
# API endpoints
location /airflow/api {
rewrite ^/airflow(/.*)$ $1 break;
proxy_pass http://airflow-webserver:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Script-Name /airflow;
proxy_cookie_path / /airflow/;
}
# Static files
location /airflow/static {
rewrite ^/airflow(/.*)$ $1 break;
proxy_pass http://airflow-webserver:8080;
proxy_set_header Host $host;
proxy_cache_valid 200 1d;
add_header Cache-Control "public, immutable";
}
# Health check
location /airflow/health {
rewrite ^/airflow(/.*)$ $1 break;
proxy_pass http://airflow-webserver:8080;
proxy_set_header Host $host;
}

View File

@@ -0,0 +1,58 @@
# API Service - FastAPI with Keycloak Authentication
# Subpath: /apiservice
# Backend: apiservice:8000
location /apiservice {
# Remove /apiservice prefix before forwarding to backend
rewrite ^/apiservice(/.*)$ $1 break;
# Forward to FastAPI backend
proxy_pass http://apiservice:8000;
# Preserve original host and protocol
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
# Important: Tell FastAPI about the subpath
proxy_set_header X-Script-Name /apiservice;
# Session cookie handling
proxy_cookie_path / /apiservice/;
# WebSocket support (for future use)
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Timeouts
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
# Buffer settings
proxy_buffering off;
proxy_request_buffering off;
}
# Static files (if any)
location /apiservice/static {
rewrite ^/apiservice/static(/.*)$ /static$1 break;
proxy_pass http://apiservice:8000;
proxy_set_header Host $host;
}
# Admin panel
location /apiservice/admin {
rewrite ^/apiservice(/.*)$ $1 break;
proxy_pass http://apiservice:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Script-Name /apiservice;
proxy_cookie_path / /apiservice/;
}

View File

@@ -0,0 +1,144 @@
# Complete Nginx Configuration Example
# สำหรับ Nginx Proxy Manager - Custom Nginx Configuration
#
# วิธีใช้:
# 1. ไปที่ Nginx Proxy Manager UI (http://your-server:8021)
# 2. สร้าง Proxy Host ใหม่
# 3. กรอก Domain Names: ai.sriphat.com (หรือ domain ของคุณ)
# 4. กรอก Forward Hostname/IP: localhost (dummy, จะใช้ config ด้านล่าง)
# 5. กรอก Forward Port: 80 (dummy)
# 6. ไปที่ tab "Advanced"
# 7. Copy config ด้านล่างนี้ทั้งหมดไปวางใน "Custom Nginx Configuration"
# 8. กด Save
# ============================================================================
# API Service - FastAPI with Keycloak
# ============================================================================
location /apiservice {
rewrite ^/apiservice(/.*)$ $1 break;
proxy_pass http://apiservice:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Script-Name /apiservice;
proxy_cookie_path / /apiservice/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_buffering off;
}
# ============================================================================
# Supabase Studio - Database Management UI
# ============================================================================
location /supabase {
rewrite ^/supabase(/.*)$ $1 break;
proxy_pass http://sdp-studio:3000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_buffering off;
}
# ============================================================================
# Supabase Kong API - REST API Gateway
# ============================================================================
location /supabase-api {
rewrite ^/supabase-api(/.*)$ $1 break;
proxy_pass http://sdp-kong:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Authorization $http_authorization;
proxy_set_header apikey $http_apikey;
proxy_buffering off;
}
# ============================================================================
# Keycloak - Single Sign-On (SSO)
# ============================================================================
location /keycloak {
rewrite ^/keycloak(/.*)$ $1 break;
proxy_pass http://keycloak:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Prefix /keycloak;
proxy_cookie_path / /keycloak/;
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
}
# ============================================================================
# Apache Superset - Business Intelligence
# ============================================================================
location /superset {
rewrite ^/superset(/.*)$ $1 break;
proxy_pass http://superset:8088;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Script-Name /superset;
proxy_cookie_path / /superset/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
}
# ============================================================================
# Apache Airflow - Workflow Orchestration (Optional)
# ============================================================================
location /airflow {
rewrite ^/airflow(/.*)$ $1 break;
proxy_pass http://airflow-webserver:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Script-Name /airflow;
proxy_cookie_path / /airflow/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
# ============================================================================
# DBT Documentation (Optional)
# ============================================================================
location /dbt {
rewrite ^/dbt(/.*)$ $1 break;
proxy_pass http://dbt-docs:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# ============================================================================
# Dozzle - Docker Log Viewer & Monitoring
# ============================================================================
location /dozzle {
proxy_pass http://dozzle:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_buffering off;
proxy_read_timeout 300s;
}

View File

@@ -0,0 +1,44 @@
# DBT Documentation - Data Transformation Docs
# Subpath: /dbt
# Backend: dbt-docs:8080
location /dbt {
# Remove /dbt prefix before forwarding
rewrite ^/dbt(/.*)$ $1 break;
# Forward to DBT docs server
proxy_pass http://dbt-docs:8080;
# Preserve headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Timeouts
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
}
# Static documentation files
location /dbt/static {
rewrite ^/dbt(/.*)$ $1 break;
proxy_pass http://dbt-docs:8080;
proxy_set_header Host $host;
proxy_cache_valid 200 1d;
add_header Cache-Control "public, immutable";
}
# Catalog and manifest files
location /dbt/catalog.json {
rewrite ^/dbt(/.*)$ $1 break;
proxy_pass http://dbt-docs:8080;
proxy_set_header Host $host;
}
location /dbt/manifest.json {
rewrite ^/dbt(/.*)$ $1 break;
proxy_pass http://dbt-docs:8080;
proxy_set_header Host $host;
}

View File

@@ -0,0 +1,410 @@
server {
listen 80;
server_name dev.sriphat.com;
client_max_body_size 100M;
# redirect to ai web while wait for main protal web in the future
location = / {
return 301 /ai/;
}
location /keycloak/ {
#rewrite ^/keycloak/(.*)$ /$1 break;
proxy_pass http://keycloak:8080;
# Add WebSocket support (Necessary for version 0.5.0 and up)
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# (Optional) Disable proxy buffering for better streaming response from models
proxy_buffering off;
# (Optional) Increase max request size for large attachments and long audio messages
#client_max_body_size 20M;
proxy_read_timeout 10m;
}
# location /supabase2/ {
# #rewrite ^/supabase2/(.*)$ /$1 break;
# proxy_pass http://sdp-kong:8000;
# # Add WebSocket support (Necessary for version 0.5.0 and up)
# proxy_http_version 1.1;
# proxy_set_header Upgrade $http_upgrade;
# proxy_set_header Connection "upgrade";
# proxy_set_header Host $host;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# proxy_set_header X-Forwarded-Proto $scheme;
# # (Optional) Disable proxy buffering for better streaming response from models
# proxy_buffering off;
# # (Optional) Increase max request size for large attachments and long audio messages
# #client_max_body_size 20M;
# proxy_read_timeout 10m;
# }
# location /ai/ {
# proxy_pass http://localhost:3001/ai/;
# # Add WebSocket support (Necessary for version 0.5.0 and up)
# proxy_http_version 1.1;
# proxy_set_header Upgrade $http_upgrade;
# proxy_set_header Connection "upgrade";
# proxy_set_header Host $host;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# proxy_set_header X-Forwarded-Proto $scheme;
# # (Optional) Disable proxy buffering for better streaming response from models
# proxy_buffering off;
# # (Optional) Increase max request size for large attachments and long audio messages
# #client_max_body_size 20M;
# proxy_read_timeout 10m;
# }
# location /dashboard/ {
# proxy_pass http://localhost:8800;
# proxy_http_version 1.1;
# proxy_set_header Upgrade $http_upgrade;
# proxy_set_header Connection 'upgrade';
# proxy_set_header Host $host;
# proxy_cache_bypass $http_upgrade;
# }
# location /dashboard-dev/ {
# proxy_pass http://localhost:8801;
# proxy_http_version 1.1;
# proxy_set_header Upgrade $http_upgrade;
# proxy_set_header Connection 'upgrade';
# proxy_set_header Host $host;
# proxy_cache_bypass $http_upgrade;
# }
# location /realtime/ {
# proxy_pass http://sdp-kong:8000/realtime/; # ส่งไปที่ endpoint ของ backend supabase
# # คอนฟิกสำหรับ WebSocket
# proxy_http_version 1.1;
# proxy_set_header Upgrade $http_upgrade;
# proxy_set_header Connection "Upgrade";
# # Header สำคัญอื่นๆ
# proxy_set_header Host $host;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# # ป้องกันแชทหลุด (Timeout 1 ชั่วโมง)
# proxy_read_timeout 3600s;
# proxy_send_timeout 3600s;
# }
location /apiservice/ {
# ส่งต่อ Request ไปยัง Backend
proxy_pass http://apiservice:8040;
# การตั้งค่า Header มาตรฐาน
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Ensure the sub-path is handled correctly
proxy_set_header X-Forwarded-Prefix /apiservice;
# รองรับการ Upload ไฟล์ขนาดใหญ่ (ป้องกัน Timeout ระหว่างส่งข้อมูล)
proxy_read_timeout 300s;
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
# ปิดการพักข้อมูลใน Buffer ของ Nginx ชั่วคราวเพื่อให้การ Upload ลื่นไหลขึ้น
proxy_request_buffering off;
proxy_buffering off;
# เพิ่มเติม: รองรับ WebSocket (เผื่อ Doc หรือ API มีการใช้ Real-time)
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}
# # Superset Analytics Dashboard -- notwork
location /superset {
proxy_pass http://superset:8088;
# WebSocket support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Standard headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Superset-specific headers for sub-path support
proxy_set_header X-Forwarded-Prefix /superset;
proxy_set_header X-Script-Name /superset;
# ตัวนี้จะช่วยให้ Superset เข้าใจเรื่อง Path ในการสร้างลิงก์ Static
proxy_set_header X-Forwarded-Host $host;
# Timeout settings (for long-running queries and dashboard loading)
proxy_read_timeout 300s;
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
# Disable buffering for better streaming
proxy_buffering off;
proxy_request_buffering off;
}
location /dozzle {
# Forward to Dozzle
proxy_pass http://dozzle:8080;
# Preserve headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
# WebSocket support for real-time logs
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Timeouts for long-running log streams
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
# Disable buffering for real-time streaming
proxy_buffering off;
proxy_request_buffering off;
}
# Static assets
location /dozzle/assets {
proxy_pass http://dozzle:8080;
proxy_set_header Host $host;
proxy_cache_valid 200 1d;
add_header Cache-Control "public, immutable";
}
# API endpoints
location /dozzle/api {
proxy_pass http://dozzle:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# WebSocket support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_buffering off;
}
# Health check
location /dozzle/healthcheck {
proxy_pass http://dozzle:8080;
proxy_set_header Host $host;
access_log off;
}
# =============================================
# Superset Analytics Dashboard
# Superset routes are at root level (no single prefix)
# We proxy ALL Superset paths directly to port 8088
# =============================================
# # Redirect /superset/ and /superset to welcome page
# location = /superset/ {
# return 302 /superset/welcome/;
# }
# location = /superset {
# return 302 /superset/welcome/;
# }
# Superset views (welcome, dashboard view, explore, etc.)
# location /superset {
# proxy_pass http://localhost:8088;
# proxy_http_version 1.1;
# proxy_set_header Upgrade $http_upgrade;
# proxy_set_header Connection 'upgrade';
# proxy_set_header Host $host;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# proxy_set_header X-Forwarded-Proto $scheme;
# proxy_set_header Accept-Encoding "";
# proxy_read_timeout 300s;
# proxy_connect_timeout 300s;
# proxy_send_timeout 300s;
# # Rewrite logo brand link in HTML
# sub_filter_once off;
# sub_filter '"path":"/"' '"path":"/superset/welcome/"';
# sub_filter '"path": "/"' '"path": "/superset/welcome/"';
# }
# # Superset API, static files, auth, and all other root-level routes
# # NOTE: /dashboard/ is NOT included here - it's handled by sriphat-dashboard on port 8800
# location ~ ^/(api|static|login|logout|chart|explore|sqllab|savedqueryview|tablemodelview|tableschemaview|tabstateview|tagview|datasource|dataset|databaseview|annotationlayer|csstemplatemodelview|rowlevelsecurity|embedded|dynamic-plugins|lang|theme|healthcheck|ping|roles|users|user_info|userinfoeditview|register|registrations|resetpassword|resetmypassword|groups|list_groups|back|swagger|alert|report|actionlog)(/|$) {
# proxy_pass http://localhost:8088;
# proxy_http_version 1.1;
# proxy_set_header Upgrade $http_upgrade;
# proxy_set_header Connection 'upgrade';
# proxy_set_header Host $host;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# proxy_set_header X-Forwarded-Proto $scheme;
# proxy_set_header Accept-Encoding "";
# proxy_read_timeout 300s;
# proxy_connect_timeout 300s;
# proxy_send_timeout 300s;
# # Rewrite logo href in HTML/JS responses
# sub_filter_once off;
# sub_filter_types application/javascript;
# sub_filter '"path":"/"' '"path":"/superset/welcome/"';
# sub_filter "'path':'/'" "'path':'/superset/welcome/'";
# }
# location /aiflow/ {
# proxy_pass http://airflow-webserver:8080;
# # WebSocket support
# proxy_http_version 1.1;
# proxy_set_header Upgrade $http_upgrade;
# proxy_set_header Connection "upgrade";
# # Standard headers
# proxy_set_header Host $host;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# proxy_set_header X-Forwarded-Proto $scheme;
# # Superset-specific headers for sub-path support
# proxy_set_header X-Forwarded-Prefix /superset;
# proxy_set_header X-Script-Name /superset;
# # Timeout settings (for long-running queries and dashboard loading)
# proxy_read_timeout 300s;
# proxy_connect_timeout 300s;
# proxy_send_timeout 300s;
# # Disable buffering for better streaming
# proxy_buffering off;
# proxy_request_buffering off;
# }
# location /dbt/ {
# proxy_pass http://dbt:8080;
# # WebSocket support
# proxy_http_version 1.1;
# proxy_set_header Upgrade $http_upgrade;
# proxy_set_header Connection "upgrade";
# # Standard headers
# proxy_set_header Host $host;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# proxy_set_header X-Forwarded-Proto $scheme;
# # Superset-specific headers for sub-path support
# proxy_set_header X-Forwarded-Prefix /superset;
# proxy_set_header X-Script-Name /superset;
# # Timeout settings (for long-running queries and dashboard loading)
# proxy_read_timeout 300s;
# proxy_connect_timeout 300s;
# proxy_send_timeout 300s;
# # Disable buffering for better streaming
# proxy_buffering off;
# proxy_request_buffering off;
# }
# =============================================
# MinIO Object Storage (Server 2: 192.168.100.9)
# =============================================
# MinIO S3 API — port 9000
# Path MUST be stripped before passing to MinIO
location /minio/ {
proxy_pass http://192.168.100.9:9000/;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
proxy_connect_timeout 300;
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
client_max_body_size 1G;
proxy_request_buffering off;
proxy_buffering off;
}
# MinIO Console UI — port 9001 (NOT 9000!)
# Path MUST be stripped: /minio-console/foo → /foo
location /minio-console/ {
rewrite ^/minio-console/(.*) /$1 break;
proxy_pass http://192.168.100.9:9001;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
# WebSocket support (Console uses WebSocket for real-time updates)
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_buffering off;
chunked_transfer_encoding off;
}
# Redirect /minio-console → /minio-console/
location = /minio-console {
return 301 $scheme://$http_host/minio-console/;
}
#listen 443 ssl; # managed by sriphat
#ssl_certificate /etc/letsencrypt/live/ai.bda.co.th/fullchain.pem; # managed by Certbot
#ssl_certificate_key /etc/letsencrypt/live/ai.bda.co.th/privkey.pem; # managed by Certbot
#include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
#ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
#server {
# listen 80 default_server;
# server_name ai.bda.co.th;
# #rewrite ^/[old-page]$ https://[domain]/[new-page] permanent;
# return 301 https://$host$request_uri;
#}

View File

@@ -0,0 +1,59 @@
# Dozzle - Docker Log Viewer & Monitoring
# Subpath: /dozzle
# Backend: dozzle:8080
location /dozzle {
# Forward to Dozzle
proxy_pass http://dozzle:8080;
# Preserve headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
# WebSocket support for real-time logs
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Timeouts for long-running log streams
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
# Disable buffering for real-time streaming
proxy_buffering off;
proxy_request_buffering off;
}
# Static assets
location /dozzle/assets {
proxy_pass http://dozzle:8080;
proxy_set_header Host $host;
proxy_cache_valid 200 1d;
add_header Cache-Control "public, immutable";
}
# API endpoints
location /dozzle/api {
proxy_pass http://dozzle:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# WebSocket support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_buffering off;
}
# Health check
location /dozzle/healthcheck {
proxy_pass http://dozzle:8080;
proxy_set_header Host $host;
access_log off;
}

View File

@@ -0,0 +1,68 @@
# Keycloak - Single Sign-On (SSO)
# Subpath: /keycloak
# Backend: keycloak:8080
location /keycloak {
# Remove /keycloak prefix before forwarding
rewrite ^/keycloak(/.*)$ $1 break;
# Forward to Keycloak
proxy_pass http://keycloak:8080;
# Preserve headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
# Important for Keycloak
proxy_set_header X-Forwarded-Prefix /keycloak;
# Session and cookie handling
proxy_cookie_path / /keycloak/;
# Buffer settings
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
# Timeouts
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
}
# Admin console
location /keycloak/admin {
rewrite ^/keycloak(/.*)$ $1 break;
proxy_pass http://keycloak:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Prefix /keycloak;
proxy_cookie_path / /keycloak/;
}
# Realms
location /keycloak/realms {
rewrite ^/keycloak(/.*)$ $1 break;
proxy_pass http://keycloak:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Prefix /keycloak;
proxy_cookie_path / /keycloak/;
}
# Resources (CSS, JS, images)
location /keycloak/resources {
rewrite ^/keycloak(/.*)$ $1 break;
proxy_pass http://keycloak:8080;
proxy_set_header Host $host;
proxy_cache_valid 200 1d;
add_header Cache-Control "public, immutable";
}

View File

@@ -0,0 +1,391 @@
# Nginx Proxy Manager - Setup Guide
คู่มือการตั้งค่า Nginx Proxy Manager สำหรับ Sriphat Data Platform
## 📋 ข้อมูล Services
| Service | Subpath | Container Name | Port | Status |
|---------|---------|----------------|------|--------|
| API Service | `/apiservice` | apiservice | 8000 | ✅ Active |
| Supabase Studio | `/supabase` | sdp-studio | 3000 | ✅ Active |
| Supabase API | `/supabase-api` | sdp-kong | 8000 | ✅ Active |
| Keycloak | `/keycloak` | keycloak | 8080 | ✅ Active |
| Superset | `/superset` | superset | 8088 | ✅ Active |
| Airflow | `/airflow` | airflow-webserver | 8080 | ⚠️ Optional |
| DBT Docs | `/dbt` | dbt-docs | 8080 | ⚠️ Optional |
## 🚀 Quick Start
### 1. เข้า Nginx Proxy Manager
```
URL: http://192.168.100.9:8021
Default Login:
Email: admin@example.com
Password: changeme
```
**⚠️ เปลี่ยน password ทันทีหลัง login ครั้งแรก!**
### 2. สร้าง Proxy Host
1. คลิก **Proxy Hosts****Add Proxy Host**
2. กรอกข้อมูล:
**Tab: Details**
```
Domain Names: ai.sriphat.com
Scheme: http
Forward Hostname/IP: localhost
Forward Port: 80
```
**Tab: Advanced**
- Copy config จาก `complete-example.conf` ทั้งหมด
- Paste ลงใน **Custom Nginx Configuration**
3. กด **Save**
### 3. ตั้งค่า SSL (Production)
**Tab: SSL**
```
☑ Request a new SSL Certificate with Let's Encrypt
☑ Force SSL
☑ HTTP/2 Support
☑ HSTS Enabled
Email: your-email@example.com
```
กด **Save**
## 📝 การใช้งานแบบละเอียด
### วิธีที่ 1: ใช้ Complete Config (แนะนำ)
**ข้อดี:**
- ตั้งค่าครั้งเดียว ได้ทุก service
- ง่ายต่อการจัดการ
- Consistent configuration
**ขั้นตอน:**
1. สร้าง Proxy Host ตาม Quick Start
2. Copy `complete-example.conf` ไปวางใน Advanced tab
3. Save
### วิธีที่ 2: แยก Config แต่ละ Service
**ข้อดี:**
- ควบคุมแต่ละ service ได้อิสระ
- ง่ายต่อการ debug
- สามารถตั้งค่า SSL แยกกันได้
**ขั้นตอน:**
#### API Service
```
Domain: api.sriphat.com
Forward: apiservice:8000
Advanced: ใช้ config จาก apiservice.conf
```
#### Supabase Studio
```
Domain: supabase.sriphat.com
Forward: sdp-studio:3000
Advanced: ใช้ config จาก supabase-studio.conf
```
#### Keycloak
```
Domain: auth.sriphat.com
Forward: keycloak:8080
Advanced: ใช้ config จาก keycloak.conf
```
#### Superset
```
Domain: bi.sriphat.com
Forward: superset:8088
Advanced: ใช้ config จาก superset.conf
```
## 🔧 Configuration Details
### API Service (`/apiservice`)
**สิ่งสำคัญ:**
```nginx
proxy_set_header X-Script-Name /apiservice;
proxy_cookie_path / /apiservice/;
```
**ทำไม:**
- FastAPI ต้องรู้ว่าทำงานภายใต้ subpath
- Session cookies ต้อง scope ถูกต้อง
### Keycloak (`/keycloak`)
**สิ่งสำคัญ:**
```nginx
proxy_set_header X-Forwarded-Prefix /keycloak;
proxy_cookie_path / /keycloak/;
```
**ทำไม:**
- Keycloak ใช้ X-Forwarded-Prefix สำหรับ redirect URLs
- Authentication flow ต้องการ cookie path ที่ถูกต้อง
### Supabase Studio (`/supabase`)
**สิ่งสำคัญ:**
```nginx
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
```
**ทำไม:**
- Supabase Studio ใช้ WebSocket สำหรับ real-time features
- ต้อง support HTTP/1.1 upgrade
### Superset (`/superset`)
**สิ่งสำคัญ:**
```nginx
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
```
**ทำไม:**
- Dashboard queries อาจใช้เวลานาน
- ต้องการ timeout ที่สูงกว่าปกติ
## 🐛 Troubleshooting
### 502 Bad Gateway
**สาเหตุ:**
- Backend service ไม่ทำงาน
- Network configuration ผิด
**วิธีแก้:**
```bash
# ตรวจสอบ service
docker ps | grep <service-name>
# ตรวจสอบ network
docker network inspect shared_data_network
# ดู logs
docker logs <service-name>
docker logs nginx-proxy-manager
```
### 404 Not Found
**สาเหตุ:**
- Path rewriting ไม่ถูกต้อง
- Backend ไม่ support subpath
**วิธีแก้:**
```nginx
# ตรวจสอบ rewrite rule
rewrite ^/apiservice(/.*)$ $1 break;
# ดู nginx logs
docker exec nginx-proxy-manager tail -f /data/logs/proxy-host-*.log
```
### Redirect Loop
**สาเหตุ:**
- Cookie path ไม่ถูกต้อง
- X-Forwarded-* headers ขาดหาย
**วิธีแก้:**
```nginx
# เพิ่ม headers
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
# ตั้งค่า cookie path
proxy_cookie_path / /apiservice/;
```
### WebSocket Connection Failed
**สาเหตุ:**
- ไม่มี WebSocket headers
- HTTP version ไม่ถูกต้อง
**วิธีแก้:**
```nginx
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
```
### Session Lost After Refresh
**สาเหตุ:**
- Cookie path ไม่ match กับ URL path
- SameSite cookie policy
**วิธีแก้:**
```nginx
proxy_cookie_path / /apiservice/;
proxy_cookie_flags ~ secure samesite=lax;
```
## 🔐 Security Best Practices
### 1. เปลี่ยน Default Password
```
Settings → Users → Edit admin user
```
### 2. ใช้ HTTPS ใน Production
```
SSL Tab → Request a new SSL Certificate
☑ Force SSL
☑ HSTS Enabled
```
### 3. ตั้งค่า Access Lists
```
Access Lists → Add Access List
- Whitelist IP addresses
- Basic authentication
- Apply to sensitive services (Keycloak Admin, Superset)
```
### 4. Enable Rate Limiting
```nginx
# ใน Custom Nginx Configuration
limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;
limit_req zone=api_limit burst=20 nodelay;
```
### 5. Hide Nginx Version
```nginx
# ใน Custom Nginx Configuration
server_tokens off;
```
## 📊 Monitoring
### ดู Logs
**ใน Nginx Proxy Manager UI:**
```
Proxy Hosts → Click on host → Logs tab
```
**ใน Docker:**
```bash
# Nginx Proxy Manager logs
docker logs nginx-proxy-manager -f
# Access logs
docker exec nginx-proxy-manager tail -f /data/logs/proxy-host-*.log
# Error logs
docker exec nginx-proxy-manager tail -f /data/logs/error.log
```
### Health Checks
```bash
# ตรวจสอบ Nginx Proxy Manager
curl http://192.168.100.9:8021
# ตรวจสอบ services ผ่าน proxy
curl http://ai.sriphat.com/apiservice/docs
curl http://ai.sriphat.com/supabase
curl http://ai.sriphat.com/keycloak
```
## 🔄 Backup & Restore
### Backup Configuration
```bash
# Backup Nginx Proxy Manager data
cd /path/to/01-infra
tar -czf npm-backup-$(date +%Y%m%d).tar.gz data/
# Backup specific configs
docker exec nginx-proxy-manager tar -czf /tmp/configs.tar.gz /data/nginx
docker cp nginx-proxy-manager:/tmp/configs.tar.gz ./npm-configs-backup.tar.gz
```
### Restore Configuration
```bash
# Stop Nginx Proxy Manager
docker compose down
# Restore data
tar -xzf npm-backup-YYYYMMDD.tar.gz
# Start Nginx Proxy Manager
docker compose up -d
```
## 📚 Additional Resources
- [Nginx Proxy Manager Docs](https://nginxproxymanager.com/guide/)
- [Nginx Reverse Proxy Guide](https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/)
- [Let's Encrypt SSL](https://letsencrypt.org/getting-started/)
## 💡 Tips & Tricks
### 1. Test Config Before Applying
```bash
# Test nginx config
docker exec nginx-proxy-manager nginx -t
```
### 2. Reload Without Restart
```bash
# Reload nginx (no downtime)
docker exec nginx-proxy-manager nginx -s reload
```
### 3. View Current Config
```bash
# View active nginx config
docker exec nginx-proxy-manager cat /etc/nginx/nginx.conf
```
### 4. Debug Mode
```nginx
# เพิ่มใน Custom Nginx Configuration
error_log /data/logs/error.log debug;
```
### 5. Custom Error Pages
```nginx
# เพิ่มใน Custom Nginx Configuration
error_page 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
```
## 🎯 Production Checklist
- [ ] เปลี่ยน default admin password
- [ ] ตั้งค่า SSL certificate (Let's Encrypt)
- [ ] Enable Force SSL
- [ ] Enable HSTS
- [ ] ตั้งค่า Access Lists สำหรับ admin panels
- [ ] Enable rate limiting
- [ ] Hide server tokens
- [ ] ตั้งค่า backup schedule
- [ ] Test all services ผ่าน proxy
- [ ] Monitor logs สำหรับ errors
- [ ] Document custom configurations

View File

@@ -0,0 +1,88 @@
# Supabase Kong API Gateway - REST API
# Subpath: /supabase-api
# Backend: sdp-kong:8000
location /supabase-api {
# Remove /supabase-api prefix before forwarding
rewrite ^/supabase-api(/.*)$ $1 break;
# Forward to Kong Gateway
proxy_pass http://sdp-kong:8000;
# Preserve headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
# API key and authorization headers
proxy_set_header Authorization $http_authorization;
proxy_set_header apikey $http_apikey;
# CORS headers (if needed)
add_header Access-Control-Allow-Origin * always;
add_header Access-Control-Allow-Methods "GET, POST, PUT, DELETE, PATCH, OPTIONS" always;
add_header Access-Control-Allow-Headers "Authorization, Content-Type, apikey, X-Client-Info" always;
# Handle preflight requests
if ($request_method = 'OPTIONS') {
return 204;
}
# Timeouts for API calls
proxy_connect_timeout 30s;
proxy_send_timeout 30s;
proxy_read_timeout 30s;
# Disable buffering for streaming responses
proxy_buffering off;
}
# REST API endpoints
location /supabase-api/rest {
rewrite ^/supabase-api(/.*)$ $1 break;
proxy_pass http://sdp-kong:8000;
proxy_set_header Host $host;
proxy_set_header Authorization $http_authorization;
proxy_set_header apikey $http_apikey;
}
# Auth endpoints
location /supabase-api/auth {
rewrite ^/supabase-api(/.*)$ $1 break;
proxy_pass http://sdp-kong:8000;
proxy_set_header Host $host;
proxy_set_header Authorization $http_authorization;
proxy_set_header apikey $http_apikey;
}
# Storage endpoints
location /supabase-api/storage {
rewrite ^/supabase-api(/.*)$ $1 break;
proxy_pass http://sdp-kong:8000;
proxy_set_header Host $host;
proxy_set_header Authorization $http_authorization;
proxy_set_header apikey $http_apikey;
# Larger timeouts for file uploads
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
client_max_body_size 100M;
}
# Realtime endpoints (WebSocket)
location /supabase-api/realtime {
rewrite ^/supabase-api(/.*)$ $1 break;
proxy_pass http://sdp-kong:8000;
proxy_set_header Host $host;
proxy_set_header Authorization $http_authorization;
proxy_set_header apikey $http_apikey;
# WebSocket support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 86400;
}

View File

@@ -0,0 +1,50 @@
# Supabase Studio - Database Management UI
# Subpath: /supabase
# Backend: sdp-studio:3000
location /supabase {
# Remove /supabase prefix before forwarding
rewrite ^/supabase(/.*)$ $1 break;
# Forward to Supabase Studio
proxy_pass http://sdp-studio:3000;
# Preserve headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
# WebSocket support for real-time features
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Timeouts
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
# Disable buffering for real-time updates
proxy_buffering off;
}
# API endpoints for Studio
location /supabase/api {
rewrite ^/supabase(/.*)$ $1 break;
proxy_pass http://sdp-studio:3000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# Static assets
location /supabase/_next {
rewrite ^/supabase(/.*)$ $1 break;
proxy_pass http://sdp-studio:3000;
proxy_set_header Host $host;
proxy_cache_valid 200 1d;
add_header Cache-Control "public, immutable";
}

View File

@@ -0,0 +1,68 @@
# Apache Superset - Business Intelligence
# Subpath: /superset
# Backend: superset:8088
location /superset {
# Remove /superset prefix before forwarding
rewrite ^/superset(/.*)$ $1 break;
# Forward to Superset
proxy_pass http://superset:8088;
# Preserve headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
# Important for Superset subpath
proxy_set_header X-Script-Name /superset;
# Session cookie handling
proxy_cookie_path / /superset/;
# WebSocket support for real-time dashboards
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Timeouts (dashboards can take time to load)
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
proxy_read_timeout 300s;
# Buffer settings for large responses
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
}
# API endpoints
location /superset/api {
rewrite ^/superset(/.*)$ $1 break;
proxy_pass http://superset:8088;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Script-Name /superset;
proxy_cookie_path / /superset/;
}
# Static files
location /superset/static {
rewrite ^/superset(/.*)$ $1 break;
proxy_pass http://superset:8088;
proxy_set_header Host $host;
proxy_cache_valid 200 1d;
add_header Cache-Control "public, immutable";
}
# Superset assets
location /superset/superset {
rewrite ^/superset(/.*)$ $1 break;
proxy_pass http://superset:8088;
proxy_set_header Host $host;
proxy_set_header X-Script-Name /superset;
}

View File

@@ -40,6 +40,7 @@ services:
LOGFLARE_URL: http://sdp-analytics:4000
NEXT_PUBLIC_ENABLE_LOGS: true
NEXT_ANALYTICS_BACKEND_PROVIDER: postgres
SNIPPETS_MANAGEMENT_FOLDER: "/app/snippets"
volumes:
- ./volumes/snippets:/app/snippets:Z
- ./volumes/functions:/app/supabase/functions:Z
@@ -433,12 +434,16 @@ services:
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
## ecto
DATABASE_URL: postgresql://supabase_admin:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/_supabase
POOLER_TENANT_ID: ${POOLER_TENANT_ID} # MUST BE ADDED
POOLER_TENANT_ID: ${POOLER_TENANT_ID:-sriphat}
CLUSTER_POSTGRES: true
SECRET_KEY_BASE: ${SECRET_KEY_BASE}
VAULT_ENC_KEY: ${VAULT_ENC_KEY}
API_JWT_SECRET: ${JWT_SECRET}
METRICS_JWT_SECRET: ${JWT_SECRET}
# Ensure these are also mapped if they are in your .env
POOLER_DEFAULT_POOL_SIZE: ${POOLER_DEFAULT_POOL_SIZE}
POOLER_MAX_CLIENT_CONN: ${POOLER_MAX_CLIENT_CONN}
POOLER_DB_POOL_SIZE: ${POOLER_DB_POOL_SIZE}
networks:
- shared_data_network

View File

@@ -1,10 +0,0 @@
.env
__pycache__/
*.pyc
.venv/
venv/
.python-version
.pytest_cache/
.mypy_cache/
ruff_cache/
.windsurf/

View File

@@ -1,17 +0,0 @@
FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
WORKDIR /app
COPY requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir -r /app/requirements.txt
COPY ./app /app/app
ENV TZ=Asia/Bangkok
EXPOSE 8040
CMD ["gunicorn","-k","uvicorn.workers.UvicornWorker","app.main:app","--bind","0.0.0.0:8040","--workers","2","--access-logfile","-","--error-logfile","-"]

View File

@@ -1,44 +0,0 @@
# 03-apiservice: Custom FastAPI Service
## Build & Start
```bash
docker compose --env-file ../.env.global up --build -d
```
## Access
Internal only - access via Nginx Proxy Manager at `/apiservice`
## Admin UI
- Login: http://<domain>/apiservice/admin/
- Generate API Key: POST /apiservice/admin/api-keys/generate
## env
env that important for provision
```
## supabase
SUPABASE_DB_HOST=sdp-db
SUPABASE_DB_PORT=5432
SUPABASE_DB_USER=postgres.1
SUPABASE_DB_PASSWORD=
SUPABASE_DB_NAME=postgres
SUPABASE_DB_SSLMODE=disable
## pgsql
DB_HOST=postgres
DB_PORT=5432
DB_USER=postgres
DB_PASSWORD=
DB_NAME=postgres
DB_SSLMODE=disable
AIRBYTE_DB_NAME=airbyte
KEYCLOAK_DB_NAME=keycloack
SUPERSET_DB_NAME=superset
#TEMPORAL_DB_NAME=temporal
## api
ROOT_PATH=/apiservice
APP_NAME=APIsService
ADMIN_SECRET_KEY=
ADMIN_USERNAME=admin
ADMIN_PASSWORD=
```

View File

@@ -1,283 +0,0 @@
from __future__ import annotations
from fastapi import HTTPException, Request, status
from fastapi.staticfiles import StaticFiles
from sqladmin import Admin, ModelView
from sqladmin.authentication import AuthenticationBackend
from starlette.responses import HTMLResponse, RedirectResponse
from starlette.datastructures import URL
from sqlalchemy.orm import sessionmaker
from wtforms import BooleanField, SelectField, StringField
from wtforms.validators import Optional
from app.core.config import settings
from app.db.engine import engine
from app.db.models import ApiClient, ApiKey
from app.security.api_key import generate_api_key, get_prefix, hash_api_key
class AdminAuth(AuthenticationBackend):
async def login(self, request: Request) -> bool:
form = await request.form()
username = form.get("username")
password = form.get("password")
if username == settings.ADMIN_USERNAME and password == settings.ADMIN_PASSWORD:
request.session.update({"admin": True})
return True
return False
async def logout(self, request: Request) -> bool:
request.session.clear()
return True
async def authenticate(self, request: Request) -> bool:
return bool(request.session.get("admin"))
class ApiClientAdmin(ModelView, model=ApiClient):
column_list = [ApiClient.id, ApiClient.name, ApiClient.is_active]
async def insert_model(self, request: Request, data: dict) -> ApiClient:
obj: ApiClient = await super().insert_model(request, data)
plain_key = generate_api_key()
db = sessionmaker(bind=engine, autoflush=False, autocommit=False)()
try:
api_key = ApiKey(
client_id=obj.id,
name="auto",
key_prefix=get_prefix(plain_key),
key_hash=hash_api_key(plain_key),
permissions=[],
is_active=True,
)
db.add(api_key)
db.commit()
db.refresh(api_key)
request.session["generated_api_key"] = {
"client_id": obj.id,
"client_name": obj.name,
"key_id": api_key.id,
"api_key": plain_key,
}
finally:
db.close()
return obj
class ApiKeyAdmin(ModelView, model=ApiKey):
column_list = [ApiKey.id, ApiKey.client_id, ApiKey.name, ApiKey.is_active, ApiKey.permissions]
form_excluded_columns = [ApiKey.key_hash, ApiKey.key_prefix, ApiKey.created_at]
form_extra_fields = {
"plain_key": StringField("Plain Key", validators=[Optional()]),
"permissions_csv": StringField("Permissions (comma)", validators=[Optional()]),
"endpoint_path": SelectField("Endpoint", choices=[], validators=[Optional()]),
"perm_read": BooleanField("Read (GET)"),
"perm_write": BooleanField("Write (POST/PATCH)"),
"perm_delete": BooleanField("Delete (DELETE)"),
}
async def on_model_change(self, data: dict, model: ApiKey, is_created: bool, request: Request) -> None:
plain_key = data.get("plain_key")
if not plain_key and is_created:
plain_key = generate_api_key()
if plain_key:
model.key_prefix = get_prefix(plain_key)
model.key_hash = hash_api_key(plain_key)
if is_created:
request.state.generated_api_key_plain = plain_key
permissions: list[str] = []
endpoint_path = data.get("endpoint_path")
if endpoint_path:
if data.get("perm_read"):
permissions.append(f"{endpoint_path}:read")
if data.get("perm_write"):
permissions.append(f"{endpoint_path}:write")
if data.get("perm_delete"):
permissions.append(f"{endpoint_path}:delete")
permissions_csv = data.get("permissions_csv")
if permissions_csv is not None:
perms = [p.strip() for p in permissions_csv.split(",") if p.strip()]
permissions.extend(perms)
if permissions:
seen: set[str] = set()
deduped: list[str] = []
for p in permissions:
if p not in seen:
seen.add(p)
deduped.append(p)
model.permissions = deduped
async def after_model_change(self, data: dict, model: ApiKey, is_created: bool, request: Request) -> None:
if not is_created:
return
plain_key = getattr(request.state, "generated_api_key_plain", None)
if not plain_key:
return
request.session["generated_api_key"] = {
"client_id": model.client_id,
"client_name": str(getattr(model, "client", "")) if getattr(model, "client", None) else "",
"key_id": model.id,
"api_key": plain_key,
}
def mount_admin(app):
auth_backend = AdminAuth(secret_key=settings.ADMIN_SECRET_KEY)
class CustomAdmin(Admin):
def get_save_redirect_url(
self, request: Request, form, model_view: ModelView, obj
):
if (
getattr(model_view, "model", None) in (ApiClient, ApiKey)
and request.session.get("generated_api_key")
):
root_path = request.scope.get("root_path") or ""
return URL(f"{root_path}/admin/generated-api-key")
return super().get_save_redirect_url(
request=request,
form=form,
model_view=model_view,
obj=obj,
)
admin = CustomAdmin(
app=app,
engine=engine,
authentication_backend=auth_backend,
title="My Service Management",
base_url="/admin",
)
openapi = app.openapi()
paths = openapi.get("paths") or {}
endpoint_choices: list[tuple[str, str]] = []
for path in sorted(paths.keys()):
if not path.startswith("/api/"):
continue
methods = paths.get(path) or {}
available = sorted([m.upper() for m in methods.keys()])
label = f"{path} [{' '.join(available)}]" if available else path
endpoint_choices.append((path, label))
ApiKeyAdmin.form_extra_fields["endpoint_path"].kwargs["choices"] = endpoint_choices
SessionLocal = sessionmaker(bind=engine, autoflush=False, autocommit=False)
admin.add_view(ApiClientAdmin)
admin.add_view(ApiKeyAdmin)
@app.get("/admin/generated-api-key")
async def _admin_generated_api_key(request: Request):
if not request.session.get("admin"):
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
key_info = request.session.pop("generated_api_key", None)
root_path = request.scope.get("root_path") or ""
clients_url = f"{root_path}/admin/{ApiClientAdmin.identity}/list"
if not key_info:
return HTMLResponse(
f"<h2>No API key to display</h2><p>The API key was already shown or expired.</p><p><a href=\"{clients_url}\">Back to clients</a></p>",
status_code=200,
)
client_name = key_info.get("client_name", "")
client_id = key_info.get("client_id", "")
key_id = key_info.get("key_id", "")
api_key = key_info.get("api_key", "")
return HTMLResponse(
(
"<h2>API key generated</h2>"
"<p>Copy this API key now. You won't be able to view it again.</p>"
f"<p><b>Client</b>: {client_name} (ID: {client_id})</p>"
f"<p><b>Key ID</b>: {key_id}</p>"
f"<pre style=\"padding:12px;border:1px solid #ddd;background:#f7f7f7;\">{api_key}</pre>"
f"<p><a href=\"{clients_url}\">Back to clients</a></p>"
),
status_code=200,
)
@app.get("/admin/clients/{client_id}/generate-api-key")
async def _admin_generate_api_key_get(
request: Request,
client_id: int,
permissions: str = "",
name: str | None = None,
):
if not request.session.get("admin"):
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
perms = [p.strip() for p in permissions.split(",") if p.strip()]
plain_key = generate_api_key()
db = SessionLocal()
try:
client = db.get(ApiClient, client_id)
if not client:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Client not found")
api_key = ApiKey(
client_id=client_id,
name=name,
key_prefix=get_prefix(plain_key),
key_hash=hash_api_key(plain_key),
permissions=perms,
is_active=True,
)
db.add(api_key)
db.commit()
db.refresh(api_key)
return {"key_id": api_key.id, "api_key": plain_key, "permissions": perms}
finally:
db.close()
@app.post("/admin/api-keys/generate")
async def _admin_generate_api_key(
request: Request,
client_id: int,
permissions: str = "",
name: str | None = None,
):
if not request.session.get("admin"):
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
perms = [p.strip() for p in permissions.split(",") if p.strip()]
plain_key = generate_api_key()
db = SessionLocal()
try:
client = db.get(ApiClient, client_id)
if not client:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Client not found")
api_key = ApiKey(
client_id=client_id,
name=name,
key_prefix=get_prefix(plain_key),
key_hash=hash_api_key(plain_key),
permissions=perms,
is_active=True,
)
db.add(api_key)
db.commit()
db.refresh(api_key)
return {"key_id": api_key.id, "api_key": plain_key, "permissions": perms}
finally:
db.close()

View File

@@ -1,127 +0,0 @@
from __future__ import annotations
import logging
from typing import Annotated
from zoneinfo import ZoneInfo
from fastapi import APIRouter, Depends
from sqlalchemy.dialects.postgresql import insert
from sqlalchemy.orm import Session
from app.api.v1.schemas import FeedCheckpointIn
from app.core.config import settings
from app.db.models import RawOpdCheckpoint
from app.security.dependencies import get_db, get_supabase_db, require_permission
from app.utils.supabase_client import SupabaseAPIError, upsert_to_supabase_sync
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v1")
PERM_FEED_CHECKPOINT_WRITE = "/api/v1/feed/checkpoint:write"
PERM_FEED_CHECKPOINT_WRITE_LEGACY = "feed.checkpoint:write"
def _to_tz(dt):
if dt is None:
return None
if dt.tzinfo is None:
return dt.replace(tzinfo=ZoneInfo(settings.TIMEZONE))
return dt.astimezone(ZoneInfo(settings.TIMEZONE))
def _to_iso(dt):
"""Convert datetime to ISO 8601 string for Supabase API."""
if dt is None:
return None
return dt.isoformat()
@router.post("/feed/checkpoint")
def upsert_feed_checkpoint(
payload: list[FeedCheckpointIn],
db: Annotated[Session, Depends(get_db)],
):
rows = []
supabase_rows = []
#clean_data = payload.model_dump(exclude_none=True)
for item in payload:
# Prepare data for local database 'default' if item.id is None else
row = {
"id": item.id,
"hn": item.hn,
"vn": item.vn,
"location": item.location,
"type": item.type,
"timestamp_in": _to_tz(item.timestamp_in),
"timestamp_out": _to_tz(item.timestamp_out),
"waiting_time": item.waiting_time,
"bu": item.bu,
}
if item.id is None:
del(row["id"])
rows.append(row)
# Prepare data for Supabase API (convert datetime to ISO string) 'default' if item.id is None else
supabase_row = {
"id": item.id,
"hn": item.hn,
"vn": item.vn,
"location": item.location,
"type": item.type,
"timestamp_in": _to_iso(_to_tz(item.timestamp_in)),
"timestamp_out": _to_iso(_to_tz(item.timestamp_out)),
"waiting_time": item.waiting_time,
"bu": item.bu,
}
if item.id is None:
del(supabase_row["id"])
supabase_rows.append(supabase_row)
# Insert/update to local database
stmt = insert(RawOpdCheckpoint).values(rows)
update_cols = {
"id": stmt.excluded.id,
"type": stmt.excluded.type,
"timestamp_in": stmt.excluded.timestamp_in,
"timestamp_out": stmt.excluded.timestamp_out,
"waiting_time": stmt.excluded.waiting_time,
"bu": stmt.excluded.bu,
}
stmt = stmt.on_conflict_do_update(
index_elements=[RawOpdCheckpoint.hn, RawOpdCheckpoint.vn, RawOpdCheckpoint.location, RawOpdCheckpoint.timestamp_in],
set_=update_cols,
)
result = db.execute(stmt)
db.commit()
# Send data to Supabase via API call
supabase_result = None
supabase_error = None
try:
logger.info(f"Sending {len(supabase_rows)} records to Supabase API")
supabase_result = upsert_to_supabase_sync(
table="raw_opd_checkpoint",
data=supabase_rows,
on_conflict="hn,vn,location,timestamp_in",
)
logger.info(f"Successfully sent data to Supabase: {supabase_result.get('status_code')}")
except SupabaseAPIError as e:
logger.error(f"Failed to send data to Supabase: {str(e)}")
supabase_error = str(e)
except Exception as e:
logger.error(f"Unexpected error sending data to Supabase: {str(e)}")
supabase_error = f"Unexpected error: {str(e)}"
return {
"upserted": len(rows),
"rowcount": result.rowcount,
"supabase": {
"success": supabase_result is not None,
"result": supabase_result,
"error": supabase_error,
},
}

View File

@@ -1,15 +0,0 @@
from datetime import datetime
from pydantic import BaseModel
class FeedCheckpointIn(BaseModel):
id: int | None = None
hn: int
vn: int
location: str
type: str
timestamp_in: datetime
timestamp_out: datetime | None = None
waiting_time: int | None = None
bu: str | None = None

View File

@@ -1,35 +0,0 @@
from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings):
model_config = SettingsConfigDict(env_file=".env", extra="ignore")
APP_NAME: str = "APIsService"
DB_HOST: str
DB_PORT: int = 5432
DB_USER: str
DB_PASSWORD: str
DB_NAME: str
DB_SSLMODE: str = "disable"
SUPABASE_DB_HOST: str
SUPABASE_DB_PORT: int = 5432
SUPABASE_DB_USER: str
SUPABASE_DB_PASSWORD: str
SUPABASE_DB_NAME: str
SUPABASE_DB_SSLMODE: str = "disable"
SUPABASE_API_URL: str
SUPABASE_API_KEY: str
ROOT_PATH: str = ""
TIMEZONE: str = "Asia/Bangkok"
ADMIN_SECRET_KEY: str
ADMIN_USERNAME: str
ADMIN_PASSWORD: str
settings = Settings()

View File

@@ -1,5 +0,0 @@
from sqlalchemy.orm import DeclarativeBase
class Base(DeclarativeBase):
pass

View File

@@ -1,35 +0,0 @@
from urllib.parse import quote_plus
from sqlalchemy import create_engine
from app.core.config import settings
def build_db_url() -> str:
user = quote_plus(settings.DB_USER)
password = quote_plus(settings.DB_PASSWORD)
host = settings.DB_HOST
port = settings.DB_PORT
db = quote_plus(settings.DB_NAME)
return (
f"postgresql+psycopg://{user}:{password}@{host}:{port}/{db}"
f"?sslmode={quote_plus(settings.DB_SSLMODE)}"
)
def build_supabase_db_url() -> str:
user = quote_plus(settings.SUPABASE_DB_USER)
password = quote_plus(settings.SUPABASE_DB_PASSWORD)
host = settings.SUPABASE_DB_HOST
port = settings.SUPABASE_DB_PORT
db = quote_plus(settings.SUPABASE_DB_NAME)
return (
f"postgresql+psycopg://{user}:{password}@{host}:{port}/{db}"
f"?sslmode={quote_plus(settings.SUPABASE_DB_SSLMODE)}"
)
engine = create_engine(build_db_url(), pool_pre_ping=True)
supabase_engine = create_engine(build_supabase_db_url(), pool_pre_ping=True)

View File

@@ -1,13 +0,0 @@
from sqlalchemy import text
from app.db.base import Base
from app.db.engine import engine
def init_db() -> None:
# with engine.begin() as conn:
# conn.execute(text("CREATE SCHEMA IF NOT EXISTS fastapi"))
# conn.execute(text("CREATE SCHEMA IF NOT EXISTS operationbi"))
# Base.metadata.create_all(bind=conn)
pass

View File

@@ -1,74 +0,0 @@
from __future__ import annotations
from datetime import datetime
from sqlalchemy import BigInteger, Boolean, DateTime, ForeignKey, Integer, String, Text, UniqueConstraint, func
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.db.base import Base
class RawOpdCheckpoint(Base):
__tablename__ = "raw_opd_checkpoint"
__table_args__ = (
UniqueConstraint("hn", "vn", "location", name="uq_raw_opd_checkpoint_hn_vn_location"),
{"schema": "rawdata"},
)
id: Mapped[int] = mapped_column(BigInteger, primary_key=True)
hn: Mapped[int] = mapped_column(BigInteger, nullable=False)
vn: Mapped[int] = mapped_column(BigInteger, nullable=False)
location: Mapped[str] = mapped_column(Text, nullable=False)
type: Mapped[str] = mapped_column(String(64), nullable=False)
timestamp_in: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
timestamp_out: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
waiting_time: Mapped[int | None] = mapped_column(Integer, nullable=True)
bu: Mapped[str | None] = mapped_column(String(128), nullable=True)
class ApiClient(Base):
__tablename__ = "api_client"
__table_args__ = {"schema": "fastapi"}
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
name: Mapped[str] = mapped_column(String(128), unique=True, nullable=False)
is_active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
api_keys: Mapped[list[ApiKey]] = relationship(
back_populates="client",
cascade="all, delete-orphan",
passive_deletes=True,
)
def __str__(self) -> str:
client_id = getattr(self, "id", None)
if client_id is None:
return self.name
return f"{self.name} ({client_id})"
def __repr__(self) -> str:
return str(self)
class ApiKey(Base):
__tablename__ = "api_key"
__table_args__ = {"schema": "fastapi"}
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
client_id: Mapped[int] = mapped_column(
ForeignKey("fastapi.api_client.id", ondelete="CASCADE"), nullable=False
)
name: Mapped[str | None] = mapped_column(String(128), nullable=True)
key_prefix: Mapped[str] = mapped_column(String(12), nullable=False)
key_hash: Mapped[str] = mapped_column(Text, nullable=False)
permissions: Mapped[list[str]] = mapped_column(JSONB, nullable=False, default=list)
is_active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=func.now()
)
client: Mapped[ApiClient] = relationship(back_populates="api_keys")

View File

@@ -1,99 +0,0 @@
from contextlib import asynccontextmanager
from fastapi import FastAPI
from starlette.datastructures import Headers
from starlette.middleware.sessions import SessionMiddleware
from fastapi.middleware.cors import CORSMiddleware
from starlette.middleware.base import BaseHTTPMiddleware
class ForceHTTPSMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request, call_next):
# บังคับให้ FastAPI มองว่า Request ที่เข้ามาเป็น HTTPS เสมอ
# เพื่อให้ url_for() เจนลิงก์ CSS/JS เป็น https://
request.scope["scheme"] = "https"
response = await call_next(request)
return response
class ForwardedProtoMiddleware:
def __init__(self, app):
self.app = app
async def __call__(self, scope, receive, send):
if scope["type"] in {"http", "websocket"}:
headers = Headers(scope=scope)
forwarded_proto = headers.get("x-forwarded-proto")
if forwarded_proto:
proto = forwarded_proto.split(",", 1)[0].strip()
if proto:
new_scope = dict(scope)
new_scope["scheme"] = proto
return await self.app(new_scope, receive, send)
return await self.app(scope, receive, send)
# class RootPathStripMiddleware:
# def __init__(self, app, prefix: str):
# self.app = app
# self.prefix = (prefix or "").rstrip("/")
# async def __call__(self, scope, receive, send):
# if scope["type"] in {"http", "websocket"} and self.prefix:
# path = scope.get("path") or ""
# new_scope = dict(scope)
# new_scope["root_path"] = self.prefix
# if path == self.prefix or path.startswith(self.prefix + "/"):
# new_path = path[len(self.prefix) :]
# new_scope["path"] = new_path if new_path else "/"
# return await self.app(new_scope, receive, send)
# return await self.app(scope, receive, send)
from app.admin import mount_admin
from app.api.v1.routes import router as v1_router
from app.core.config import settings
from app.db.init_db import init_db
from fastapi.staticfiles import StaticFiles
from sqladmin import Admin
import os
import sqladmin
# รายชื่อ Origins ที่อนุญาตให้ยิง API มาหาเราได้
origins = [
"http://localhost:80400", # สำหรับตอนพัฒนา Frontend
"https://ai.sriphat.com", # Domain หลักของคุณ
"http://ai.sriphat.com",
]
@asynccontextmanager
async def lifespan(_: FastAPI):
init_db()
yield
print(settings.ROOT_PATH, flush=True)
sqladmin_dir = os.path.dirname(sqladmin.__file__)
statics_path = os.path.join(sqladmin_dir, "statics")
app = FastAPI(title=settings.APP_NAME, root_path=settings.ROOT_PATH, lifespan=lifespan)
#if settings.ROOT_PATH:
# app.add_middleware(RootPathStripMiddleware, prefix=settings.ROOT_PATH)
app.add_middleware(ForceHTTPSMiddleware)
app.add_middleware(SessionMiddleware, secret_key=settings.ADMIN_SECRET_KEY)
app.add_middleware(ForwardedProtoMiddleware)
app.include_router(v1_router)
app.mount("/admin/statics", StaticFiles(directory=statics_path), name="admin_statics")
app.mount("/apiservice/admin/statics", StaticFiles(directory=statics_path), name="proxy_admin_statics")
app.add_middleware(
CORSMiddleware,
allow_origins=origins, # หรือ ["*"] ถ้าต้องการอนุญาตทั้งหมด (ไม่แนะนำใน production)
allow_credentials=True, # สำคัญมาก! ต้องเป็น True ถ้าหน้า Admin/API มีการใช้ Cookies/Sessions
allow_methods=["*"], # อนุญาตทุก HTTP Method (GET, POST, PUT, DELETE, etc.)
allow_headers=["*"], # อนุญาตทุก Headers
)
mount_admin(app)

View File

@@ -1,22 +0,0 @@
import secrets
import bcrypt
def generate_api_key(prefix_len: int = 8, token_bytes: int = 32) -> str:
prefix = secrets.token_urlsafe(prefix_len)[:prefix_len]
token = secrets.token_urlsafe(token_bytes)
return f"{prefix}.{token}"
def get_prefix(api_key: str) -> str:
return api_key.split(".", 1)[0]
def hash_api_key(api_key: str) -> str:
hashed = bcrypt.hashpw(api_key.encode("utf-8"), bcrypt.gensalt())
return hashed.decode("utf-8")
def verify_api_key(api_key: str, api_key_hash: str) -> bool:
return bcrypt.checkpw(api_key.encode("utf-8"), api_key_hash.encode("utf-8"))

View File

@@ -1,66 +0,0 @@
from typing import Annotated
from collections.abc import Sequence
from fastapi import Depends, HTTPException, Request, status
from sqlalchemy import select
from sqlalchemy.orm import Session, sessionmaker
from app.db.engine import engine, supabase_engine
from app.db.models import ApiKey
from app.security.api_key import get_prefix, verify_api_key
SessionLocal = sessionmaker(bind=engine, autoflush=False, autocommit=False)
SupabaseSessionLocal = sessionmaker(bind=supabase_engine, autoflush=False, autocommit=False)
def get_db():
db = SessionLocal()
try:
yield db
finally:
db.close()
def get_supabase_db():
db = SupabaseSessionLocal()
try:
yield db
finally:
db.close()
def get_bearer_token(request: Request) -> str:
auth = request.headers.get("authorization")
if not auth:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Missing Authorization")
parts = auth.split(" ", 1)
if len(parts) != 2 or parts[0].lower() != "bearer" or not parts[1].strip():
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid Authorization")
return parts[1].strip()
def require_permission(permission: str | Sequence[str]):
def _dep(
token: Annotated[str, Depends(get_bearer_token)],
db: Annotated[Session, Depends(get_db)],
) -> ApiKey:
prefix = get_prefix(token)
stmt = select(ApiKey).where(ApiKey.key_prefix == prefix, ApiKey.is_active.is_(True))
api_key = db.execute(stmt).scalar_one_or_none()
if not api_key:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid API key")
if not verify_api_key(token, api_key.key_hash):
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid API key")
allowed = set(api_key.permissions or [])
required = [permission] if isinstance(permission, str) else list(permission)
if not any(p in allowed for p in required):
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Permission denied")
return api_key
return _dep

View File

@@ -1,37 +0,0 @@
services:
apiservice:
build: .
container_name: apiservice
env_file:
- ../.env.global
environment:
- TZ=${TZ:-Asia/Bangkok}
- DB_HOST=${DB_HOST}
- DB_PORT=${DB_PORT}
- DB_USER=${DB_USER}
- DB_PASSWORD=${DB_PASSWORD}
- DB_NAME=${DB_NAME}
- DB_SSLMODE=${DB_SSLMODE}
- ROOT_PATH=${ROOT_PATH}
- APP_NAME=${APP_NAME}
- ADMIN_SECRET_KEY=${ADMIN_SECRET_KEY}
- ADMIN_USERNAME=${ADMIN_USERNAME}
- ADMIN_PASSWORD=${ADMIN_PASSWORD}
networks:
- shared_data_network
restart: unless-stopped
volumes:
- ./app:/app/app
- .env:/app/.env
ports:
- 0.0.0.0:8040:8040
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8040/apiservice/docs', timeout=5).read()"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
networks:
shared_data_network:
external: true

View File

@@ -1,16 +0,0 @@
fastapi==0.115.8
uvicorn==0.34.0
gunicorn==23.0.0
SQLAlchemy==2.0.38
psycopg==3.2.5
pydantic==2.10.6
pydantic-settings==2.7.1
psycopg[binary]
sqladmin==0.20.1
itsdangerous==2.2.0
bcrypt==4.3.0
python-multipart==0.0.20
httpx==0.28.1
WTForms
#==3.2.1

View File

@@ -32,3 +32,26 @@ ADMIN_PASSWORD=your-admin-password
# API Key Encryption (for storing encrypted keys in DB)
API_KEY_ENC_SECRET=your-encryption-secret-key-here
# Debug settings (set to true for detailed logging, false for production)
DEBUG_AUTH=false
# Keycloak Authentication (for web pages)
KEYCLOAK_SERVER_URL=http://keycloak:8080
KEYCLOAK_REALM=master
KEYCLOAK_CLIENT_ID=apiservice
KEYCLOAK_CLIENT_SECRET=your-keycloak-client-secret
KEYCLOAK_REDIRECT_URI=http://localhost:8040/apiservice/auth/callback
# Airflow Integration
AIRFLOW_API_URL=http://airflow-webserver:8080
AIRFLOW_API_TOKEN=your-airflow-api-token
AIRFLOW_DAG_ID_FINANCE=process_finance_excel
# MinIO Object Storage (server 2: 192.168.100.9)
# ใช้ service account sp_service_ac (ไม่ใช้ root credentials)
MINIO_ENDPOINT=192.168.100.9:9000
MINIO_SVC_ACCESS_KEY=sp_service_ac
MINIO_SVC_SECRET_KEY=your-minio-service-account-secret
MINIO_BUCKET_FINANCE=finance
MINIO_USE_SSL=false

View File

@@ -8,3 +8,5 @@ venv/
.mypy_cache/
ruff_cache/
.windsurf/
data/uploads/*
!data/uploads/.gitkeep

View File

@@ -0,0 +1,334 @@
# Airflow Integration Guide
คู่มือการตั้งค่าและใช้งาน Airflow Integration สำหรับ Finance Excel Upload
## 📋 Overview
เมื่อ upload ไฟล์ Excel ผ่านหน้า `/data-management/finance` ระบบจะ:
1. บันทึกไฟล์ลง `/data/uploads/`
2. สร้าง record ใน database (`fastapi.upload_history`)
3. **Trigger Airflow DAG** `process_finance_excel` (พร้อม retry 3 ครั้ง, รอ 10 วิ)
4. Airflow จะ process ไฟล์และอัพเดท status กลับมา
---
## 🔧 Configuration
### 1. สร้าง Airflow API Token
```bash
# เข้า Airflow container
docker exec -it airflow-webserver bash
# สร้าง API token
airflow users create \
--username apiservice \
--firstname API \
--lastname Service \
--role Admin \
--email apiservice@sriphat.com \
--password your-secure-password
# หรือใช้ CLI สร้าง token
python -c "from airflow.api.auth.backend.basic_auth import BasicAuth; print(BasicAuth().get_token('apiservice', 'your-secure-password'))"
```
**หรือใช้ Airflow UI:**
1. เข้า `http://ai.sriphat.com/airflow`
2. ไปที่ **Security****List Users**
3. เลือก user → **Edit**
4. Copy **API Token** (หรือสร้างใหม่)
### 2. อัพเดท Environment Variables
แก้ไข `.env` ใน `03-apiservice/`:
```bash
# Airflow Integration
AIRFLOW_API_URL=http://airflow-webserver:8080
AIRFLOW_API_TOKEN=your-airflow-api-token-here
AIRFLOW_DAG_ID_FINANCE=process_finance_excel
```
### 3. Restart API Service
```bash
cd 03-apiservice
docker compose restart
```
---
## 📊 Database Schema Updates
### New Fields in `fastapi.upload_history`:
| Field | Type | Description |
|-------|------|-------------|
| `airflow_dag_run_id` | String | DAG run ID จาก Airflow |
| `airflow_state` | String | State: queued, running, success, failed |
| `processing_started_at` | DateTime | เวลาเริ่ม process |
| `processing_completed_at` | DateTime | เวลาเสร็จ |
| `error_message` | Text | Error message (ถ้ามี) |
### Migration
```bash
# Restart apiservice เพื่อสร้าง columns ใหม่
cd 03-apiservice
docker compose down
docker compose up -d
```
---
## 🚀 How It Works
### Upload Flow:
```
1. User uploads Excel file
2. API saves file to /data/uploads/
3. Create UploadHistory record (status: pending)
4. Trigger Airflow DAG (with retry logic)
├─ Attempt 1: Try trigger
├─ Wait 10 seconds
├─ Attempt 2: Retry
├─ Wait 10 seconds
└─ Attempt 3: Final retry
5. Update record:
- Success: status=processing, airflow_dag_run_id=xxx
- Failure: status=error, error_message=xxx
6. Return response to frontend
```
### Airflow DAG Flow:
```
validate_file
process_data
load_to_database
update_status
```
---
## 📝 API Endpoints
### Upload File (Triggers Airflow)
```http
POST /data-management/finance/upload
Content-Type: multipart/form-data
file: [Excel file]
description: "Optional description"
```
**Response:**
```json
{
"success": true,
"message": "File 'finance.xlsx' uploaded successfully",
"upload_id": "upload_20260313_173000",
"filename": "20260313_173000_finance.xlsx",
"airflow_triggered": true,
"dag_run_id": "manual__2026-03-13T10:30:00+00:00",
"error": null
}
```
### Get Upload History
```http
GET /data-management/finance/uploads
```
**Response:**
```json
[
{
"id": "upload_20260313_173000",
"filename": "finance.xlsx",
"filepath": "/data/uploads/20260313_173000_finance.xlsx",
"uploaded_at": "2026-03-13T17:30:00",
"description": "Q1 2026 Finance Data",
"status": "processing",
"uploaded_by": "admin",
"airflow_dag_run_id": "manual__2026-03-13T10:30:00+00:00",
"airflow_state": "running",
"processing_started_at": "2026-03-13T17:30:05",
"processing_completed_at": null,
"error_message": null
}
]
```
---
## 🔍 Monitoring
### 1. Check Upload Status
```bash
# ดู logs ของ API Service
docker logs apiservice -f | grep "Airflow"
# ตัวอย่าง logs:
# Triggering Airflow DAG (attempt 1/3)
# Airflow DAG triggered successfully: manual__2026-03-13T10:30:00+00:00
```
### 2. Check Airflow DAG
1. เข้า Airflow UI: `http://ai.sriphat.com/airflow`
2. ไปที่ **DAGs**`process_finance_excel`
3. ดู **DAG Runs** และ **Task Instances**
### 3. Check Database
```sql
-- ดู upload history พร้อม Airflow status
SELECT
upload_id,
filename,
status,
airflow_dag_run_id,
airflow_state,
processing_started_at,
processing_completed_at,
error_message
FROM fastapi.upload_history
ORDER BY uploaded_at DESC
LIMIT 10;
```
---
## 🐛 Troubleshooting
### Issue: "Failed to trigger Airflow after 3 attempts"
**สาเหตุ:**
- Airflow API Token ไม่ถูกต้อง
- Airflow webserver ไม่ทำงาน
- Network connection ล้มเหลว
**วิธีแก้:**
```bash
# 1. ตรวจสอบ Airflow webserver
docker ps | grep airflow-webserver
# 2. ตรวจสอบ network
docker network inspect shared_data_network
# 3. ทดสอบ API token
curl -X GET \
http://airflow-webserver:8080/api/v1/dags \
-H "Authorization: Bearer YOUR_TOKEN"
# 4. ดู logs
docker logs airflow-webserver -f
docker logs apiservice -f
```
### Issue: DAG ไม่ปรากฏใน Airflow UI
**สาเหตุ:**
- DAG file มี syntax error
- DAG ถูก pause
- Airflow scheduler ไม่ทำงาน
**วิธีแก้:**
```bash
# 1. ตรวจสอบ DAG file
docker exec airflow-webserver airflow dags list | grep process_finance_excel
# 2. Test DAG
docker exec airflow-webserver airflow dags test process_finance_excel
# 3. Unpause DAG
docker exec airflow-webserver airflow dags unpause process_finance_excel
# 4. ตรวจสอบ scheduler
docker logs airflow-scheduler -f
```
### Issue: DAG triggered แต่ไม่ทำงาน
**สาเหตุ:**
- Volume mount ไม่ถูกต้อง (Airflow เข้าถึง `/data/uploads` ไม่ได้)
- Dependencies ขาดหาย (pandas, openpyxl)
**วิธีแก้:**
```bash
# 1. ตรวจสอบ volume
docker exec airflow-worker ls -la /data/uploads
# 2. ติดตั้ง dependencies
docker exec airflow-worker pip install pandas openpyxl
# หรือเพิ่มใน docker-compose.yaml:
# _PIP_ADDITIONAL_REQUIREMENTS: "pandas openpyxl"
```
---
## 🔐 Security Notes
1. **API Token Security:**
- เก็บ token ใน `.env` (ไม่ commit ลง git)
- ใช้ token ที่มี scope จำกัด (ไม่ใช่ admin token)
- Rotate token เป็นระยะ
2. **File Access:**
- Airflow ต้องมี read access ไปยัง `/data/uploads`
- ตรวจสอบ file permissions
3. **Network:**
- API Service และ Airflow ต้องอยู่ใน `shared_data_network`
- ใช้ internal network (ไม่ expose Airflow API ออกภายนอก)
---
## 📚 References
- [Airflow REST API Documentation](https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html)
- [Airflow Authentication](https://airflow.apache.org/docs/apache-airflow/stable/security/api.html)
- [FastAPI Background Tasks](https://fastapi.tiangolo.com/tutorial/background-tasks/)
---
## 🎯 Next Steps
1. **Implement DAG Logic:**
- แก้ไข `05-airflow/dags/process_finance_excel.py`
- เพิ่ม data transformation logic
- เพิ่ม database loading logic
2. **Add Callback:**
- สร้าง endpoint `/api/v1/airflow/callback/{upload_id}`
- Airflow จะ callback เมื่อเสร็จ/ล้มเหลว
- อัพเดท `processing_completed_at` และ `airflow_state`
3. **Frontend Updates:**
- เพิ่ม real-time status polling
- แสดง Airflow state badges
- Link ไปยัง Airflow DAG run
4. **Testing:**
- ทดสอบ upload file
- ตรวจสอบ Airflow trigger
- ตรวจสอบ retry logic
- ทดสอบ error handling

View File

@@ -0,0 +1,138 @@
# Changes — 2026-06-04
## สรุป
วันนี้เพิ่ม 2 feature ใหม่ใน `03-apiservice`:
1. **VOC Data endpoint** — รับข้อมูลข้อร้องเรียน (Voice of Customer) จาก programmatic client
2. **API Management page** — หน้าจัดการ API clients/keys ด้วย Keycloak admin auth แทน SQLAdmin basic auth เดิม
---
## Feature 1 — VOC Data Endpoint
### ไฟล์ที่แก้ไข
| ไฟล์ | การเปลี่ยนแปลง |
|------|----------------|
| `app/api/v1/schemas.py` | เพิ่ม `VocDataIn` schema |
| `app/db/models.py` | เพิ่ม `RawVocData` model (table: `rawdata.raw_voc_data`) |
| `app/api/v1/routes.py` | เพิ่ม `POST /api/v1/voc-data` endpoint |
### Endpoint
```
POST /api/v1/voc-data
Authorization: Bearer <api-key> (permission required: voc.data:write)
Content-Type: application/json
```
**Request body** (batch array):
```json
[
{
"date": "2026-06-04",
"topic": "บริการพยาบาล",
"sub_topic": "ความรวดเร็ว",
"level": "3",
"depart_id": "OPD01",
"dep_name": "ผู้ป่วยนอก"
}
]
```
**Response:**
```json
{
"inserted": 1,
"rowcount": 1,
"supabase": { "success": true, "result": {...}, "error": null }
}
```
### Database Table
```sql
CREATE TABLE rawdata.raw_voc_data (
id BIGSERIAL PRIMARY KEY,
date DATE NOT NULL,
topic VARCHAR(200) NOT NULL,
sub_topic VARCHAR(200) NOT NULL,
level VARCHAR(50) NOT NULL,
depart_id VARCHAR(50) NOT NULL,
dep_name VARCHAR(200),
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
```
- `id` เป็น BIGSERIAL auto-increment — server generate เอง, client ไม่ต้องส่ง
- ทุก record ที่ส่งมาจะ INSERT เพิ่มเสมอ (ไม่มี upsert/on_conflict)
- สร้าง table อัตโนมัติจาก `Base.metadata.create_all()` ตอน startup
- Dual-write ไปยัง Supabase (`raw_voc_data` table) เหมือน endpoint อื่นๆ
### Deploy status
✅ Deploy แล้ว บน server .8 — table `rawdata.raw_voc_data` ถูกสร้างแล้ว
---
## Feature 2 — API Management Page (Keycloak auth)
### ปัญหาเดิม
SQLAdmin panel (`/admin/`) ใช้ basic auth (username/password จาก `.env`) แยกต่างหากจาก Keycloak ซึ่งเป็น auth system หลักของระบบ
### การแก้ไข
ปิด SQLAdmin และสร้างหน้าจัดการ API keys ใหม่ที่ใช้ Keycloak admin auth แทน
### ไฟล์ที่แก้ไข
| ไฟล์ | การเปลี่ยนแปลง |
|------|----------------|
| `app/main.py` | ลบ SQLAdmin imports/mounts (`sqladmin`, statics, `mount_admin`) |
| `app/admin.py` | Comment out `/admin` redirect route |
| `app/middleware/auth_middleware.py` | เพิ่ม `/api-management`, `/admin/users` ใน `PROTECTED_PATHS` |
| `app/routes/pages.py` | เพิ่ม `GET /api-management` route |
| `app/templates/index.html` | เปลี่ยน link จาก `/admin/``/api-management` |
### ไฟล์ใหม่
| ไฟล์ | คำอธิบาย |
|------|---------|
| `app/routes/admin_api_keys.py` | REST endpoints สำหรับจัดการ API clients/keys (Keycloak admin auth) |
| `app/templates/api_management.html` | หน้าจัดการ API clients และ keys |
### Endpoints ใหม่ (ทั้งหมดต้องการ Keycloak admin role)
| Method | Path | คำอธิบาย |
|--------|------|---------|
| GET | `/admin/api-keys/clients` | List ทุก API client พร้อม nested keys |
| POST | `/admin/api-keys/clients` | สร้าง API client ใหม่ |
| POST | `/admin/api-keys/generate` | สร้าง API key (คืน plaintext ครั้งเดียว) |
| POST | `/admin/api-keys/{id}/regenerate` | Regenerate key (คืน plaintext ครั้งเดียว) |
| PATCH | `/admin/api-keys/{id}/toggle` | Toggle is_active |
### Features ของหน้า `/api-management`
- Stats: จำนวน clients, total keys, active keys
- สร้าง API Client พร้อมกำหนดชื่อ
- สร้าง API Key พร้อมกำหนด permissions เป็น JSON array
- แสดง plaintext key ใน modal ครั้งเดียวหลัง generate/regenerate พร้อมปุ่ม Copy
- Activate/Deactivate key
- เข้าถึงได้ที่ `https://ai.sriphat.com/apiservice/api-management` (ต้อง login ด้วย Keycloak admin account)
### Deploy status
⏳ ยังไม่ deploy — รอ review ก่อน
---
## Blockers / สิ่งที่ค้างอยู่
- **Airflow API token** ยังไม่ได้ config → Finance upload จะ set status=error หลัง upload สำเร็จ (ไฟล์อัปขึ้น MinIO ได้ แต่ trigger DAG ไม่ได้)
- **VOC API key** — ยังต้องสร้าง ApiClient + ApiKey ที่มี permission `voc.data:write` สำหรับ client ที่จะส่งข้อมูล (ทำได้หลัง deploy Feature 2)
---
## 🧠 Decision & Lesson
_(เขียนเอง)_

View File

@@ -0,0 +1,325 @@
# Keycloak Setup Guide for API Service Authentication
## Prerequisites
1. Keycloak must be running at `http://localhost:8085`
2. PostgreSQL database must have `keycloak` database created
3. API Service must have Keycloak configuration in `.env`
## Step 1: Restart Infrastructure to Create Keycloak Database
```bash
cd 01-infra
docker compose down
docker compose --env-file ../.env.global up -d
```
Wait for Keycloak to initialize (check logs):
```bash
docker logs keycloak -f
```
Look for: `Keycloak ... started`
## Step 2: Access Keycloak Admin Console
1. Open browser: `http://localhost:8085/admin`
2. Login credentials:
- **Username**: `admin`
- **Password**: `admin_secret_pass_2026`
## Step 3: Create Client for API Service
### 3.1 Navigate to Clients
1. In left sidebar, click **Clients**
2. Click **Create client** button
### 3.2 General Settings
- **Client type**: `OpenID Connect`
- **Client ID**: `apiservice`
- Click **Next**
### 3.3 Capability Config
- **Client authentication**: `ON` (toggle to enable)
- **Authorization**: `OFF`
- **Authentication flow**:
- ✅ Standard flow
- ✅ Direct access grants
- ❌ Implicit flow (uncheck)
- ❌ Service accounts roles (uncheck)
- Click **Next**
### 3.4 Login Settings
Fill in the following URLs:
**Root URL**:
```
http://localhost:8040
```
**Home URL**:
```
http://localhost:8040/apiservice/
```
**Valid redirect URIs** (add both):
```
http://localhost:8040/apiservice/auth/callback
https://ai.sriphat.com/apiservice/auth/callback
```
**Valid post logout redirect URIs** (add both):
```
http://localhost:8040/apiservice/
https://ai.sriphat.com/apiservice/
```
**Web origins** (add both):
```
http://localhost:8040
https://ai.sriphat.com
```
Click **Save**
### 3.5 Get Client Secret
1. Go to **Credentials** tab
2. Copy the **Client secret** value
3. Save it - you'll need it for the `.env` file
## Step 4: Configure API Service
### 4.1 Create/Update `.env` file in `03-apiservice`
```bash
cd ../03-apiservice
```
Create `.env` file with:
```bash
# Application
APP_NAME=APIsService
ROOT_PATH=/apiservice
# Timezone
TIMEZONE=Asia/Bangkok
# PostgreSQL Database
DB_HOST=postgres
DB_PORT=5432
DB_USER=postgres
DB_PASSWORD=Secure_Hospital_Pass_2026
DB_NAME=postgres
DB_SSLMODE=prefer
# Supabase (if not using, keep dummy values)
SUPABASE_DB_HOST=localhost
SUPABASE_DB_PORT=5432
SUPABASE_DB_USER=postgres
SUPABASE_DB_PASSWORD=dummy
SUPABASE_DB_NAME=postgres
SUPABASE_DB_SSLMODE=disable
SUPABASE_API_URL=http://localhost:8000
SUPABASE_API_KEY=dummy
# Admin Authentication
ADMIN_SECRET_KEY=apiservice_admin_secret_2026
ADMIN_USERNAME=admin
ADMIN_PASSWORD=change_me_2026
# API Key Encryption
API_KEY_ENC_SECRET=dev_encryption_secret_2026
# Keycloak Authentication
KEYCLOAK_SERVER_URL=http://keycloak:8080
KEYCLOAK_REALM=master
KEYCLOAK_CLIENT_ID=apiservice
KEYCLOAK_CLIENT_SECRET=<PASTE_YOUR_CLIENT_SECRET_HERE>
KEYCLOAK_REDIRECT_URI=http://localhost:8040/apiservice/auth/callback
```
**Important**: Replace `<PASTE_YOUR_CLIENT_SECRET_HERE>` with the actual secret from Step 3.5
## Step 5: Create Test User in Keycloak
### 5.1 Create User
1. In Keycloak Admin, go to **Users**
2. Click **Add user**
3. Fill in:
- **Username**: `testuser`
- **Email**: `test@sriphat.local`
- **First name**: `Test`
- **Last name**: `User`
- **Email verified**: `ON` (toggle)
4. Click **Create**
### 5.2 Set Password
1. Go to **Credentials** tab
2. Click **Set password**
3. Fill in:
- **Password**: `password`
- **Password confirmation**: `password`
- **Temporary**: `OFF` (toggle off)
4. Click **Save**
5. Confirm by clicking **Save password**
## Step 6: Rebuild and Restart API Service
```bash
cd 03-apiservice
docker compose down
docker compose --env-file ../.env up --build -d
```
Check logs:
```bash
docker logs apiservice -f
```
Look for successful startup without errors.
## Step 7: Test Authentication Flow
### 7.1 Access Landing Page
1. Open browser: `http://localhost:8040/apiservice/`
2. You should be redirected to Keycloak login page
### 7.2 Login
1. Enter credentials:
- **Username**: `testuser`
- **Password**: `password`
2. Click **Sign In**
### 7.3 Verify Success
1. You should be redirected back to the landing page
2. You should see your name in the top-right corner: "👤 Test User"
3. You should see a **Logout** button
### 7.4 Test Protected Pages
Try accessing:
- `/apiservice/` - Landing page (requires auth)
- `/apiservice/docs` - API docs (requires auth)
- `/apiservice/data-management/finance` - Finance page (requires auth)
All should work without redirecting to login again.
### 7.5 Test API Endpoints (Should NOT require Keycloak auth)
```bash
# API endpoints use API Key authentication, not Keycloak
curl -X GET http://localhost:8040/apiservice/api/v1/health
```
This should work without Keycloak authentication.
### 7.6 Test Logout
1. Click **Logout** button
2. You should be logged out from Keycloak
3. Accessing any protected page should redirect to login again
## Troubleshooting
### Error: "access_denied" in Keycloak logs
**Cause**: Client not configured correctly or client secret mismatch
**Solution**:
1. Verify client ID is exactly `apiservice`
2. Verify client secret in `.env` matches Keycloak
3. Verify redirect URIs are correct
4. Restart API service after changing `.env`
### Error: "SessionMiddleware must be installed"
**Cause**: Middleware order is incorrect
**Solution**: Already fixed in `app/main.py` - SessionMiddleware is added after WebAuthenticationMiddleware
### Error: "Invalid redirect_uri"
**Cause**: Redirect URI not in Keycloak's allowed list
**Solution**:
1. Go to Keycloak → Clients → apiservice → Settings
2. Add the exact redirect URI to "Valid redirect URIs"
3. Make sure it matches `KEYCLOAK_REDIRECT_URI` in `.env`
### Error: Connection refused to Keycloak
**Cause**: Keycloak not running or wrong URL
**Solution**:
1. Check Keycloak is running: `docker ps | grep keycloak`
2. Verify `KEYCLOAK_SERVER_URL=http://keycloak:8080` (use container name, not localhost)
3. Verify both services are on same Docker network
## Production Deployment
For production deployment:
1. **Use HTTPS**:
```bash
KEYCLOAK_REDIRECT_URI=https://ai.sriphat.com/apiservice/auth/callback
```
2. **Update Keycloak Client**:
- Add production redirect URIs
- Remove localhost URIs
3. **Secure Cookies**:
- SessionMiddleware should use `secure=True, httponly=True, samesite='lax'`
4. **Use Nginx Proxy Manager**:
- Configure SSL certificates
- Set up proper reverse proxy rules
5. **Environment Variables**:
- Use Docker secrets or external secret management
- Never commit `.env` with real secrets to git
## Security Notes
1. **Client Secret**: Keep this secret! Never commit to git
2. **Session Secret**: Use strong random value for `ADMIN_SECRET_KEY`
3. **HTTPS**: Always use HTTPS in production
4. **Token Expiration**: Configure appropriate token lifetimes in Keycloak
5. **CORS**: Restrict `allow_origins` to known domains only
## Architecture
```
User Browser
↓ (1) Access protected page
API Service (FastAPI)
↓ (2) Check session - not authenticated
↓ (3) Redirect to Keycloak login
Keycloak (OAuth2/OIDC)
↓ (4) User enters credentials
↓ (5) Redirect back with auth code
API Service
↓ (6) Exchange code for tokens
↓ (7) Store user info in session
↓ (8) Redirect to original page
User sees protected page with user info
```
## API vs Web Authentication
| Type | Authentication Method | Protected Routes |
|------|----------------------|------------------|
| **Web Pages** | Keycloak (OAuth2/OIDC) | `/`, `/docs`, `/data-management/*` |
| **API Endpoints** | API Key (Bearer Token) | `/api/v1/*` |
| **Admin Panel** | SQLAdmin (Basic Auth) | `/admin/*` |
This separation allows:
- Human users to login via Keycloak for web UI
- Applications/services to use API Keys for programmatic access
- Admins to manage API keys via separate admin panel

View File

@@ -0,0 +1,226 @@
# Web Pages Documentation
## Overview
The API service now includes web pages for easy access to all platform services and data management functionality.
## Pages
### 1. Landing Page (`/`)
Main dashboard with navigation to all services:
- **Supabase**: PostgreSQL database with REST API
- **API Docs**: Interactive API documentation
- **Airflow**: Workflow orchestration
- **Airbyte**: Data integration and ETL
- **Data Management**: File upload and processing (with submenu)
- **DBT**: Data transformation
- **Superset**: Business intelligence and visualization
### 2. Finance Excel Upload (`/data-management/finance`)
Upload and process financial Excel files with the following features:
**Upload Form:**
- Drag-and-drop or click to select Excel files (.xlsx, .xls)
- Optional description field
- File validation and size display
**Upload Processing:**
- Files saved to `/data/uploads/` with timestamp prefix
- Unique filename generation to prevent conflicts
- Automatic Airflow job triggering (when configured)
**Upload History:**
- Real-time list of all uploaded files
- Status tracking: pending, processing, success, error
- Expandable accordion for logs and error details
- Auto-refresh every 10 seconds
## API Endpoints
### Page Routes
```
GET / - Landing page
GET /data-management/finance - Finance upload page
POST /data-management/finance/upload - Upload file endpoint
GET /data-management/finance/uploads - Get all uploads
GET /data-management/finance/uploads/{upload_id} - Get specific upload
```
### Upload Endpoint
**POST** `/data-management/finance/upload`
**Form Data:**
- `file`: Excel file (.xlsx or .xls)
- `description`: Optional description text
**Response:**
```json
{
"success": true,
"message": "File 'example.xlsx' uploaded successfully",
"upload_id": "upload_20260306_174500",
"filename": "20260306_174500_example.xlsx"
}
```
**Error Response:**
```json
{
"detail": "Invalid file type. Only .xlsx and .xls files are allowed."
}
```
## File Storage
Uploaded files are stored in:
```
/data/uploads/YYYYMMDD_HHMMSS_original_filename.xlsx
```
Example:
```
/data/uploads/20260306_174530_finance_report.xlsx
```
## Airflow Integration
The upload endpoint includes a placeholder for Airflow integration. To enable:
1. **Configure Airflow endpoint** in `app/routes/pages.py`:
```python
airflow_url = "http://airflow-webserver:8080/api/v1/dags/{dag_id}/dagRuns"
```
2. **Set DAG ID** for the finance processing job
3. **Uncomment the Airflow trigger code** in `trigger_airflow_job()` function
4. **Configure authentication** (username/password or API token)
### Example Airflow Integration
```python
async def trigger_airflow_job(filepath: str, upload_id: str) -> str:
import httpx
airflow_url = "http://airflow-webserver:8080/api/v1/dags/finance_processing/dagRuns"
headers = {"Content-Type": "application/json"}
auth = ("airflow", "airflow")
payload = {
"conf": {
"filepath": filepath,
"upload_id": upload_id
}
}
async with httpx.AsyncClient() as client:
response = await client.post(
airflow_url,
json=payload,
headers=headers,
auth=auth
)
response.raise_for_status()
result = response.json()
return result["dag_run_id"]
```
## Upload Status Tracking
Upload records include:
- `id`: Unique identifier
- `filename`: Original filename
- `filepath`: Full path to saved file
- `uploaded_at`: ISO timestamp
- `description`: User-provided description
- `status`: pending | processing | success | error
- `job_id`: Airflow DAG run ID (when triggered)
- `logs`: Error logs or processing information
## Database Storage
Currently uses in-memory storage (`uploads_db` list). For production:
1. **Create database table:**
```sql
CREATE TABLE uploads (
id VARCHAR(100) PRIMARY KEY,
filename VARCHAR(255) NOT NULL,
filepath VARCHAR(500) NOT NULL,
uploaded_at TIMESTAMP NOT NULL,
description TEXT,
status VARCHAR(20) NOT NULL,
job_id VARCHAR(100),
logs TEXT,
created_at TIMESTAMP DEFAULT NOW()
);
```
2. **Replace in-memory storage** with SQLAlchemy model
3. **Add database queries** in route handlers
## Access URLs
- **Local**: http://localhost:8040/apiservice/
- **Production**: https://ai.sriphat.com/apiservice/
## Security Considerations
1. **File validation**: Only .xlsx and .xls files allowed
2. **Filename sanitization**: Spaces replaced with underscores
3. **Unique filenames**: Timestamp prefix prevents conflicts
4. **File size limits**: Configure in nginx/FastAPI if needed
5. **Authentication**: Add authentication middleware for production
## Future Enhancements
1. **Database persistence**: Replace in-memory storage
2. **File preview**: Show Excel data preview before processing
3. **Batch upload**: Support multiple file uploads
4. **Download processed files**: Allow downloading results
5. **User management**: Track uploads by user
6. **Email notifications**: Notify on processing completion
7. **Scheduled jobs**: Automatic periodic processing
8. **Data validation**: Validate Excel structure before processing
## Troubleshooting
### Upload fails with 500 error
- Check `/data/uploads/` directory exists and is writable
- Verify volume mount in docker-compose.yml
- Check container logs: `docker logs apiservice`
### Files not persisting after container restart
- Ensure volume mount is configured: `./data/uploads:/data/uploads`
- Check file permissions on host directory
### Airflow job not triggering
- Verify Airflow is accessible from container
- Check network connectivity: `docker network inspect shared_data_network`
- Verify Airflow credentials and DAG ID
- Check logs in upload history
## Development
To add new data management pages:
1. Create HTML template in `app/templates/`
2. Add route in `app/routes/pages.py`
3. Update landing page menu in `index.html`
4. Add submenu item if needed
Example:
```python
@router.get("/data-management/hr", response_class=HTMLResponse)
async def hr_page(request: Request):
return templates.TemplateResponse(
"data_management_hr.html",
{"request": request, "root_path": settings.ROOT_PATH}
)
```

View File

@@ -91,10 +91,11 @@ def mount_admin(app):
admin.add_view(ApiClientAdmin)
admin.add_view(ApiKeyAdmin)
@app.get("/admin")
async def _admin_redirect(request: Request):
root_path = request.scope.get("root_path") or ""
return RedirectResponse(url=f"{root_path}/admin/")
# SQLAdmin /admin route disabled — replaced by Keycloak-protected /api-management page
# @app.get("/admin")
# async def _admin_redirect(request: Request):
# root_path = request.scope.get("root_path") or ""
# return RedirectResponse(url=f"{root_path}/admin/")
@app.post("/admin/api-keys/generate")
async def _admin_generate_api_key(

View File

@@ -9,9 +9,9 @@ from fastapi import APIRouter, Depends
from sqlalchemy.dialects.postgresql import insert
from sqlalchemy.orm import Session
from app.api.v1.schemas import FeedCheckpointIn, FeedWaitingTimeIn
from app.api.v1.schemas import FeedCheckpointIn, FeedWaitingTimeIn, PatientAppointmentIn, VocDataIn
from app.core.config import settings
from app.db.models import RawOpdCheckpoint, RawWaitingTime
from app.db.models import RawOpdCheckpoint, RawWaitingTime, PatientAppointment, RawVocData
from app.security.dependencies import get_db, require_permission
from app.utils.supabase_client import SupabaseAPIError, upsert_to_supabase_sync
@@ -21,6 +21,8 @@ router = APIRouter(prefix="/api/v1")
PERM_FEED_CHECKPOINT_WRITE = "feed.checkpoint:write"
PERM_FEED_OLD_CHECKPOINT_WRITE = "feed.old-checkpoint:write"
PERM_FEED_PATIENT_APPOINTMENT_WRITE = "feed.patient-appointment:write"
PERM_VOC_DATA_WRITE = "voc.data:write"
def _to_tz(dt):
@@ -220,3 +222,124 @@ def upsert_opd_checkpoint(
"error": supabase_error,
},
}
@router.post("/feed/patient-appointment")
def upsert_patient_appointment(
payload: list[PatientAppointmentIn],
_: Annotated[object, Depends(require_permission(PERM_FEED_PATIENT_APPOINTMENT_WRITE))],
db: Annotated[Session, Depends(get_db)],
):
rows = []
supabase_rows = []
for item in payload:
# Prepare data for local database
row = {
"hn": item.hn,
"txn": item.txn,
"date": item.date, #+' 0:00:00'
"time": _to_tz(datetime.strptime(item.date.strftime("%Y-%m-%d")+' '+item.time,'%Y-%m-%d %H:%M:%S')),
"doctor_code": item.doctor_code,
"period": item.period,
"appointment_type": item.appointment_type,
"updated_at": datetime.now(ZoneInfo(settings.TIMEZONE)),
}
rows.append(row)
# Prepare data for Supabase API (convert datetime to ISO string)
supabase_row = {
"hn": item.hn,
"txn": item.txn,
"date": _to_iso(_to_tz(datetime.strptime(item.date.strftime("%Y-%m-%d")+' 00:00:00+07:00','%Y-%m-%d %H:%M:%S%z'))),
"time": _to_iso(_to_tz(datetime.strptime(item.date.strftime("%Y-%m-%d")+' '+item.time+'+07:00','%Y-%m-%d %H:%M:%S%z'))),
"doctor_code": item.doctor_code,
"period": item.period,
"appointment_type": item.appointment_type,
"updated_at": datetime.now(ZoneInfo(settings.TIMEZONE)).isoformat(),
}
supabase_rows.append(supabase_row)
# Insert/update to local database
stmt = insert(PatientAppointment).values(rows)
update_cols = {
"txn": stmt.excluded.txn,
"doctor_code": stmt.excluded.doctor_code,
"period": stmt.excluded.period,
"appointment_type": stmt.excluded.appointment_type,
"updated_at": stmt.excluded.updated_at,
}
stmt = stmt.on_conflict_do_update(
index_elements=[PatientAppointment.hn, PatientAppointment.date, PatientAppointment.time],
set_=update_cols,
)
result = db.execute(stmt)
db.commit()
# Send data to Supabase via API call
supabase_result = None
supabase_error = None
try:
logger.info(f"Sending {len(supabase_rows)} patient appointment records to Supabase API")
supabase_result = upsert_to_supabase_sync(
table="patient_appointment",
data=supabase_rows,
on_conflict="hn,date,time",
)
logger.info(f"Successfully sent patient appointment data to Supabase: {supabase_result.get('status_code')}")
except SupabaseAPIError as e:
logger.error(f"Failed to send patient appointment data to Supabase: {str(e)}")
supabase_error = str(e)
except Exception as e:
logger.error(f"Unexpected error sending patient appointment data to Supabase: {str(e)}")
supabase_error = f"Unexpected error: {str(e)}"
return {
"upserted": len(rows),
"rowcount": result.rowcount,
"supabase": {
"success": supabase_result is not None,
"result": supabase_result,
"error": supabase_error,
},
}
@router.post("/voc-data")
def insert_voc_data(
payload: list[VocDataIn],
_: Annotated[object, Depends(require_permission(PERM_VOC_DATA_WRITE))],
db: Annotated[Session, Depends(get_db)],
):
rows = [r.model_dump() for r in payload]
stmt = insert(RawVocData).values(rows)
result = db.execute(stmt)
db.commit()
supabase_rows = [{**r, "date": r["date"].isoformat()} for r in rows]
supabase_result = None
supabase_error = None
try:
logger.info(f"Sending {len(supabase_rows)} VOC records to Supabase API")
supabase_result = upsert_to_supabase_sync(table="raw_voc_data", data=supabase_rows)
logger.info(f"Successfully sent VOC data to Supabase: {supabase_result.get('status_code')}")
except SupabaseAPIError as e:
logger.error(f"Failed to send VOC data to Supabase: {str(e)}")
supabase_error = str(e)
except Exception as e:
logger.error(f"Unexpected error sending VOC data to Supabase: {str(e)}")
supabase_error = f"Unexpected error: {str(e)}"
return {
"inserted": len(rows),
"rowcount": result.rowcount,
"supabase": {
"success": supabase_result is not None,
"result": supabase_result,
"error": supabase_error,
},
}

View File

@@ -1,4 +1,4 @@
from datetime import datetime
from datetime import datetime, time, date
from pydantic import BaseModel
@@ -27,3 +27,22 @@ class FeedCheckpointIn(BaseModel):
timestamp_out: datetime | None = None
waiting_time: int | None = None
bu: str | None = None
class PatientAppointmentIn(BaseModel):
hn: str
txn: int | None = None
date: date
time: str
doctor_code: str | None = None
period: str | None = None
appointment_type: str | None = None
class VocDataIn(BaseModel):
date: date
topic: str
sub_topic: str
level: str
depart_id: str
dep_name: str | None = None

View File

@@ -33,5 +33,27 @@ class Settings(BaseSettings):
API_KEY_ENC_SECRET: str | None = None
# Debug settings
DEBUG_AUTH: bool = False # Set to True to enable detailed authentication logging
# Keycloak Authentication (for web pages only)
KEYCLOAK_SERVER_URL: str = "http://keycloak:8080"
KEYCLOAK_REALM: str = "master"
KEYCLOAK_CLIENT_ID: str = "apiservice"
KEYCLOAK_CLIENT_SECRET: str = ""
KEYCLOAK_REDIRECT_URI: str = "http://localhost:8040/apiservice/auth/callback"
# Airflow Integration
AIRFLOW_API_URL: str = "http://airflow-webserver:8080"
AIRFLOW_API_TOKEN: str = ""
AIRFLOW_DAG_ID_FINANCE: str = "process_finance_excel"
# MinIO Object Storage
MINIO_ENDPOINT: str = "192.168.100.9:9000"
MINIO_ACCESS_KEY: str = ""
MINIO_SECRET_KEY: str = ""
MINIO_BUCKET_FINANCE: str = "finance"
MINIO_USE_SSL: bool = False
settings = Settings()

View File

@@ -1,12 +1,54 @@
from sqlalchemy import text
from sqlalchemy.orm import Session
import logging
from app.db.base import Base
from app.db.engine import engine
from app.models.user import User, Role # Import models to ensure they're registered
logger = logging.getLogger(__name__)
def init_db() -> None:
"""Initialize database schemas and tables"""
with engine.begin() as conn:
# Create schemas
conn.execute(text("CREATE SCHEMA IF NOT EXISTS fastapi"))
conn.execute(text("CREATE SCHEMA IF NOT EXISTS operationbi"))
conn.execute(text("CREATE SCHEMA IF NOT EXISTS rawdata"))
# Create all tables
Base.metadata.create_all(bind=conn)
logger.info("Database schemas and tables created")
# Seed default roles
seed_roles()
def seed_roles() -> None:
"""Seed default roles if they don't exist"""
from app.db.session import SessionLocal
db = SessionLocal()
try:
roles_data = [
{"name": "admin", "description": "Full system access - can manage users and access all features"},
{"name": "operation", "description": "Data management access - can upload and manage data"}
]
for role_data in roles_data:
existing = db.query(Role).filter(Role.name == role_data["name"]).first()
if not existing:
role = Role(**role_data)
db.add(role)
logger.info(f"Created role: {role_data['name']}")
else:
logger.info(f"Role already exists: {role_data['name']}")
db.commit()
logger.info("Role seeding completed")
except Exception as e:
logger.error(f"Error seeding roles: {e}")
db.rollback()
finally:
db.close()

View File

@@ -1,8 +1,8 @@
from __future__ import annotations
from datetime import datetime
from datetime import date, datetime
from sqlalchemy import BigInteger, Boolean, DateTime, ForeignKey, Integer, String, Text, UniqueConstraint, func
from sqlalchemy import BigInteger, Boolean, Date, DateTime, ForeignKey, Integer, String, Text, UniqueConstraint, func
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column, relationship
@@ -46,6 +46,39 @@ class RawWaitingTime(Base):
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
class PatientAppointment(Base):
__tablename__ = "patient_appointment"
__table_args__ = (
UniqueConstraint("hn", "date", "time", name="uq_patient_appointment_hn_date_time"),
{"schema": "rawdata"},
)
id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True)
hn: Mapped[str] = mapped_column(String(50), nullable=False)
txn: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
date: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
time: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
doctor_code: Mapped[str | None] = mapped_column(String(50), nullable=True)
period: Mapped[str | None] = mapped_column(String(50), nullable=True)
appointment_type: Mapped[str | None] = mapped_column(String(100), nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
class RawVocData(Base):
__tablename__ = "raw_voc_data"
__table_args__ = {"schema": "rawdata"}
id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True)
date: Mapped[date] = mapped_column(Date, nullable=False)
topic: Mapped[str] = mapped_column(String(200), nullable=False)
sub_topic: Mapped[str] = mapped_column(String(200), nullable=False)
level: Mapped[str] = mapped_column(String(50), nullable=False)
depart_id: Mapped[str] = mapped_column(String(50), nullable=False)
dep_name: Mapped[str | None] = mapped_column(String(200), nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
class ApiClient(Base):
__tablename__ = "api_client"
__table_args__ = {"schema": "fastapi"}

View File

@@ -0,0 +1,27 @@
"""
Database session management
"""
from sqlalchemy.orm import sessionmaker, Session
from app.db.engine import engine
# Create SessionLocal class
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
def get_db() -> Session:
"""
Dependency to get database session
Usage:
@app.get("/items")
def read_items(db: Session = Depends(get_db)):
...
Yields:
Database session
"""
db = SessionLocal()
try:
yield db
finally:
db.close()

View File

@@ -1,17 +1,17 @@
from contextlib import asynccontextmanager
import logging
import os
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from fastapi.staticfiles import StaticFiles
from starlette.datastructures import Headers
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.middleware.sessions import SessionMiddleware
import sqladmin
from app.admin import mount_admin
from app.api.v1.routes import router as v1_router
from app.routes.pages import router as pages_router
from app.routes.auth import router as auth_router
from app.routes.admin_users import router as admin_users_router
from app.routes.admin_api_keys import router as admin_api_keys_router
from app.middleware.auth_middleware import WebAuthenticationMiddleware
from app.core.config import settings
from app.db.init_db import init_db
@@ -22,7 +22,6 @@ logging.basicConfig(
)
logging.getLogger("uvicorn.error").setLevel(logging.DEBUG)
logging.getLogger("uvicorn.access").setLevel(logging.INFO)
logging.getLogger("sqladmin").setLevel(logging.DEBUG)
logging.getLogger("sqlalchemy.engine").setLevel(logging.INFO)
@@ -64,9 +63,6 @@ async def lifespan(_: FastAPI):
yield
sqladmin_dir = os.path.dirname(sqladmin.__file__)
statics_path = os.path.join(sqladmin_dir, "statics")
app = FastAPI(title=settings.APP_NAME, root_path=settings.ROOT_PATH, lifespan=lifespan)
# Add exception handler to log all errors with traceback
@@ -80,8 +76,9 @@ async def global_exception_handler(request, exc):
status_code=500,
content={"detail": "Internal server error", "error": str(exc)}
)
# Middleware order is important! They execute in reverse order (LIFO)
# WebAuthenticationMiddleware needs SessionMiddleware, so SessionMiddleware must be added AFTER
app.add_middleware(ForceHTTPSMiddleware)
app.add_middleware(SessionMiddleware, secret_key=settings.ADMIN_SECRET_KEY)
app.add_middleware(ForwardedProtoMiddleware)
app.add_middleware(
CORSMiddleware,
@@ -90,7 +87,14 @@ app.add_middleware(
allow_methods=["*"],
allow_headers=["*"],
)
app.include_router(v1_router)
app.mount("/admin/statics", StaticFiles(directory=statics_path), name="admin_statics")
app.mount("/apiservice/admin/statics", StaticFiles(directory=statics_path), name="proxy_admin_statics")
mount_admin(app)
# Add web authentication middleware (protects /, /docs, /data-management/* only)
# API endpoints (/api/v1/*) continue to use API Key authentication
app.add_middleware(WebAuthenticationMiddleware)
# SessionMiddleware must be added AFTER middlewares that use it (due to LIFO execution)
app.add_middleware(SessionMiddleware, secret_key=settings.ADMIN_SECRET_KEY)
app.include_router(v1_router) # API endpoints - use API Key auth
app.include_router(pages_router) # Web pages - use Keycloak auth
app.include_router(auth_router) # Authentication routes
app.include_router(admin_users_router) # Admin user management API
app.include_router(admin_api_keys_router) # API key management - use Keycloak admin auth

View File

@@ -0,0 +1 @@
# Middleware package

View File

@@ -0,0 +1,171 @@
"""
Authentication middleware for web pages
Protects web UI routes while allowing API endpoints to use API Key auth
"""
from fastapi import Request
from fastapi.responses import RedirectResponse
from starlette.middleware.base import BaseHTTPMiddleware
from app.core.config import settings
import logging
logger = logging.getLogger(__name__)
class WebAuthenticationMiddleware(BaseHTTPMiddleware):
"""
Middleware to enforce Keycloak authentication for web pages
Protected routes (require user login):
- / (landing page)
- /docs (API documentation)
- /data-management/* (data management pages)
Excluded routes (no auth required or use different auth):
- /auth/* (authentication endpoints)
- /api/v1/* (API endpoints - use API Key auth)
- /admin/* (SQLAdmin - has its own auth)
- /apiservice/admin/statics/* (admin static files)
- /admin/statics/* (admin static files)
"""
# Routes that require user authentication
PROTECTED_PATHS = [
"/",
"/docs",
"/redoc",
"/openapi.json",
"/data-management",
"/api-management",
"/admin/users",
]
# Routes that are excluded from user authentication
EXCLUDED_PATHS = [
"/auth", # Authentication endpoints
"/api/v1", # API endpoints (use API Key)
"/admin", # Admin API endpoints (use require_role dependency)
]
async def dispatch(self, request: Request, call_next):
"""Process request and check authentication if needed"""
# Get the path without root_path prefix
path = request.url.path
# Remove root_path prefix if it exists
if settings.ROOT_PATH and path.startswith(settings.ROOT_PATH):
path = path[len(settings.ROOT_PATH):]
# Ensure path starts with /
if not path.startswith("/"):
path = "/" + path
# Check if path is excluded from authentication
is_excluded = any(path.startswith(excluded) for excluded in self.EXCLUDED_PATHS)
if is_excluded:
# Skip authentication for excluded paths (including /api/v1/*)
return await call_next(request)
# Check if path requires authentication
requires_auth = any(
path == protected or path.startswith(protected + "/")
for protected in self.PROTECTED_PATHS
)
if requires_auth:
# Check if user is authenticated
user = request.session.get("user")
if not user:
# User not authenticated - redirect to login
# Preserve the original path for redirect after login
original_path = request.url.path
# Build login URL with redirect
login_url = f"{settings.ROOT_PATH}/auth/login?redirect_to={original_path}"
logger.info(f"Unauthenticated access to {original_path}, redirecting to login")
return RedirectResponse(url=login_url, status_code=302)
# Role-based access control for specific paths
user_roles = user.get("roles", [])
# /data-management/* requires 'operation' or 'admin' role
if path.startswith("/data-management"):
if not any(role in user_roles for role in ["admin", "operation"]):
logger.warning(
f"Access denied: User {user.get('username')} "
f"(roles: {user_roles}) tried to access {path}"
)
# Return 403 Forbidden page
from fastapi.responses import HTMLResponse
return HTMLResponse(
content=f"""
<!DOCTYPE html>
<html>
<head>
<title>Access Denied</title>
<style>
body {{
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
display: flex;
align-items: center;
justify-content: center;
min-height: 100vh;
margin: 0;
}}
.container {{
background: white;
padding: 60px;
border-radius: 20px;
box-shadow: 0 20px 60px rgba(0,0,0,0.3);
text-align: center;
max-width: 500px;
}}
h1 {{ color: #ff6b6b; font-size: 48px; margin-bottom: 20px; }}
p {{ color: #666; font-size: 18px; margin: 15px 0; }}
.role-info {{
background: #f8f9fa;
padding: 20px;
border-radius: 10px;
margin: 20px 0;
}}
.required {{ color: #ff6b6b; font-weight: bold; }}
.your-roles {{ color: #667eea; font-weight: bold; }}
a {{
display: inline-block;
margin-top: 30px;
padding: 12px 30px;
background: #667eea;
color: white;
text-decoration: none;
border-radius: 8px;
font-weight: 600;
transition: all 0.3s;
}}
a:hover {{ background: #5568d3; transform: translateY(-2px); }}
</style>
</head>
<body>
<div class="container">
<h1>🚫</h1>
<h2>Access Denied</h2>
<p>You don't have permission to access this page.</p>
<div class="role-info">
<p>Required role: <span class="required">operation</span> or <span class="required">admin</span></p>
<p>Your roles: <span class="your-roles">{', '.join(user_roles) if user_roles else 'None'}</span></p>
</div>
<p>Please contact your administrator if you need access.</p>
<a href="{settings.ROOT_PATH}/">← Go to Home</a>
</div>
</body>
</html>
""",
status_code=403
)
# Continue with request
return await call_next(request)

View File

@@ -0,0 +1,7 @@
"""
Database models package
"""
from app.models.user import User, Role
from app.models.upload import UploadHistory
__all__ = ["User", "Role", "UploadHistory"]

View File

@@ -0,0 +1,33 @@
"""
Upload History model for tracking file uploads
"""
from sqlalchemy import Column, Integer, String, DateTime, Text
from sqlalchemy.sql import func
from app.db.base import Base
class UploadHistory(Base):
"""
Upload history tracking
Stores information about uploaded files and their processing status
"""
__tablename__ = "upload_history"
__table_args__ = {'schema': 'fastapi'}
id = Column(Integer, primary_key=True, index=True)
upload_id = Column(String, unique=True, index=True, nullable=False)
filename = Column(String, nullable=False)
filepath = Column(String, nullable=False)
description = Column(Text)
status = Column(String, default="pending")
job_id = Column(String)
logs = Column(Text)
uploaded_by = Column(String)
uploaded_at = Column(DateTime(timezone=True), server_default=func.now(), nullable=False)
updated_at = Column(DateTime(timezone=True), onupdate=func.now())
airflow_dag_run_id = Column(String)
airflow_state = Column(String)
processing_started_at = Column(DateTime(timezone=True))
processing_completed_at = Column(DateTime(timezone=True))
error_message = Column(Text)

View File

@@ -0,0 +1,55 @@
"""
User and Role models for local user management
Note: This is separate from Keycloak users - used for tracking and audit
"""
from sqlalchemy import Column, Integer, String, Boolean, DateTime, Table, ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy.sql import func
from app.db.base import Base
# Association table for many-to-many relationship
user_roles = Table(
'user_roles',
Base.metadata,
Column('user_id', Integer, ForeignKey('fastapi.users.id'), primary_key=True),
Column('role_id', Integer, ForeignKey('fastapi.roles.id'), primary_key=True),
schema='fastapi'
)
class User(Base):
"""
Local user record (synced from Keycloak)
Used for tracking, audit, and local permissions
"""
__tablename__ = "users"
__table_args__ = {'schema': 'fastapi'}
id = Column(Integer, primary_key=True, index=True)
keycloak_id = Column(String, unique=True, index=True, nullable=False) # Keycloak sub
username = Column(String, unique=True, index=True, nullable=False)
email = Column(String, unique=True, index=True)
full_name = Column(String)
is_active = Column(Boolean, default=True)
created_at = Column(DateTime(timezone=True), server_default=func.now())
updated_at = Column(DateTime(timezone=True), onupdate=func.now())
last_login = Column(DateTime(timezone=True))
# Relationships
roles = relationship("Role", secondary=user_roles, back_populates="users")
class Role(Base):
"""
Roles (synced from Keycloak)
"""
__tablename__ = "roles"
__table_args__ = {'schema': 'fastapi'}
id = Column(Integer, primary_key=True, index=True)
name = Column(String, unique=True, nullable=False, index=True) # admin, operation
description = Column(String)
created_at = Column(DateTime(timezone=True), server_default=func.now())
# Relationships
users = relationship("User", secondary=user_roles, back_populates="roles")

View File

@@ -0,0 +1 @@
# Routes package

View File

@@ -0,0 +1,196 @@
"""
API Client and API Key management endpoints (Admin only)
Uses Keycloak admin role authentication — same pattern as admin_users.py
"""
from typing import List
from fastapi import APIRouter, Depends, HTTPException, Request
from sqlalchemy.orm import Session
from pydantic import BaseModel
from datetime import datetime
from app.db.session import get_db
from app.db.models import ApiClient, ApiKey
from app.security.permissions import require_role, Roles
from app.security.api_key import generate_api_key, hash_api_key, encrypt_api_key, get_prefix
import logging
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/admin/api-keys", tags=["admin-api-keys"])
class ApiKeySchema(BaseModel):
id: int
name: str | None = None
key_prefix: str
permissions: list
is_active: bool
created_at: datetime
class Config:
from_attributes = True
class ApiClientSchema(BaseModel):
id: int
name: str
is_active: bool
api_keys: List[ApiKeySchema] = []
class Config:
from_attributes = True
class ApiClientCreateSchema(BaseModel):
name: str
class ApiClientUpdateSchema(BaseModel):
name: str | None = None
is_active: bool | None = None
class ApiKeyCreateSchema(BaseModel):
client_id: int
name: str | None = None
permissions: list[str] = []
class ApiKeyUpdateSchema(BaseModel):
name: str | None = None
permissions: list[str] | None = None
@router.get("/clients", response_model=List[ApiClientSchema])
async def list_clients(
db: Session = Depends(get_db),
current_user: dict = Depends(require_role(Roles.ADMIN)),
):
"""List all API clients with their keys (Admin only)"""
return db.query(ApiClient).order_by(ApiClient.id).all()
@router.patch("/clients/{client_id}", response_model=ApiClientSchema)
async def update_client(
client_id: int,
data: ApiClientUpdateSchema,
db: Session = Depends(get_db),
current_user: dict = Depends(require_role(Roles.ADMIN)),
):
"""Update API client name or active status (Admin only)"""
client = db.get(ApiClient, client_id)
if not client:
raise HTTPException(status_code=404, detail="Client not found")
if data.name is not None:
existing = db.query(ApiClient).filter(ApiClient.name == data.name, ApiClient.id != client_id).first()
if existing:
raise HTTPException(status_code=400, detail="Client name already exists")
client.name = data.name
if data.is_active is not None:
client.is_active = data.is_active
db.commit()
db.refresh(client)
logger.info(f"Admin {current_user.get('username')} updated client {client_id}")
return client
@router.post("/clients", response_model=ApiClientSchema)
async def create_client(
data: ApiClientCreateSchema,
db: Session = Depends(get_db),
current_user: dict = Depends(require_role(Roles.ADMIN)),
):
"""Create a new API client (Admin only)"""
existing = db.query(ApiClient).filter(ApiClient.name == data.name).first()
if existing:
raise HTTPException(status_code=400, detail="Client name already exists")
client = ApiClient(name=data.name, is_active=True)
db.add(client)
db.commit()
db.refresh(client)
return client
@router.post("/generate")
async def generate_key(
data: ApiKeyCreateSchema,
db: Session = Depends(get_db),
current_user: dict = Depends(require_role(Roles.ADMIN)),
):
"""Generate a new API key for a client (Admin only). Returns plaintext key once."""
client = db.get(ApiClient, data.client_id)
if not client:
raise HTTPException(status_code=404, detail="Client not found")
plain_key = generate_api_key()
api_key = ApiKey(
client_id=data.client_id,
name=data.name,
key_prefix=get_prefix(plain_key),
key_hash=hash_api_key(plain_key),
encrypted_key=encrypt_api_key(plain_key),
permissions=data.permissions,
is_active=True,
)
db.add(api_key)
db.commit()
db.refresh(api_key)
logger.info(f"Admin {current_user.get('username')} created API key {api_key.id} for client {client.name}")
return {"key_id": api_key.id, "api_key": plain_key, "key_prefix": api_key.key_prefix, "permissions": data.permissions}
@router.post("/{key_id}/regenerate")
async def regenerate_key(
key_id: int,
db: Session = Depends(get_db),
current_user: dict = Depends(require_role(Roles.ADMIN)),
):
"""Regenerate an API key — preserves permissions, returns new plaintext once (Admin only)"""
api_key = db.get(ApiKey, key_id)
if not api_key:
raise HTTPException(status_code=404, detail="API Key not found")
plain_key = generate_api_key()
api_key.key_prefix = get_prefix(plain_key)
api_key.key_hash = hash_api_key(plain_key)
api_key.encrypted_key = encrypt_api_key(plain_key)
db.commit()
db.refresh(api_key)
logger.info(f"Admin {current_user.get('username')} regenerated API key {key_id}")
return {"key_id": api_key.id, "api_key": plain_key, "key_prefix": api_key.key_prefix, "permissions": api_key.permissions}
@router.patch("/{key_id}", response_model=ApiKeySchema)
async def update_key(
key_id: int,
data: ApiKeyUpdateSchema,
db: Session = Depends(get_db),
current_user: dict = Depends(require_role(Roles.ADMIN)),
):
"""Update API key name or permissions (Admin only)"""
api_key = db.get(ApiKey, key_id)
if not api_key:
raise HTTPException(status_code=404, detail="API Key not found")
if data.name is not None:
api_key.name = data.name
if data.permissions is not None:
api_key.permissions = data.permissions
db.commit()
db.refresh(api_key)
logger.info(f"Admin {current_user.get('username')} updated API key {key_id}")
return api_key
@router.patch("/{key_id}/toggle")
async def toggle_key(
key_id: int,
db: Session = Depends(get_db),
current_user: dict = Depends(require_role(Roles.ADMIN)),
):
"""Toggle API key active/inactive (Admin only)"""
api_key = db.get(ApiKey, key_id)
if not api_key:
raise HTTPException(status_code=404, detail="API Key not found")
api_key.is_active = not api_key.is_active
db.commit()
return {"key_id": key_id, "is_active": api_key.is_active}

View File

@@ -0,0 +1,156 @@
"""
User and Role management endpoints (Admin only)
Note: This manages local user records synced from Keycloak
API endpoints (/api/v1/*) are NOT affected and continue using API Key authentication
"""
from typing import List
from fastapi import APIRouter, Depends, HTTPException, Request
from sqlalchemy.orm import Session
from pydantic import BaseModel
from datetime import datetime
from app.db.session import get_db
from app.models.user import User, Role
from app.security.permissions import require_role, Roles
import logging
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/admin/users", tags=["admin-users"])
# Pydantic schemas
class RoleSchema(BaseModel):
id: int
name: str
description: str | None = None
class Config:
from_attributes = True
class UserSchema(BaseModel):
id: int
keycloak_id: str
username: str
email: str | None = None
full_name: str | None = None
is_active: bool
roles: List[RoleSchema] = []
last_login: datetime | None = None
created_at: datetime
class Config:
from_attributes = True
class UserCreateSchema(BaseModel):
keycloak_id: str
username: str
email: str | None = None
full_name: str | None = None
class UserUpdateSchema(BaseModel):
email: str | None = None
full_name: str | None = None
is_active: bool | None = None
role_ids: List[int] | None = None
@router.get("/", response_model=List[UserSchema])
async def list_users(
db: Session = Depends(get_db),
current_user: dict = Depends(require_role(Roles.ADMIN))
):
"""List all users (Admin only)"""
users = db.query(User).all()
return users
@router.get("/{user_id}", response_model=UserSchema)
async def get_user(
user_id: int,
db: Session = Depends(get_db),
current_user: dict = Depends(require_role(Roles.ADMIN))
):
"""Get user by ID (Admin only)"""
user = db.query(User).filter(User.id == user_id).first()
if not user:
raise HTTPException(status_code=404, detail="User not found")
return user
@router.post("/", response_model=UserSchema)
async def create_user(
user_data: UserCreateSchema,
db: Session = Depends(get_db),
current_user: dict = Depends(require_role(Roles.ADMIN))
):
"""Create new user record (Admin only)"""
# Check if user already exists
existing = db.query(User).filter(User.keycloak_id == user_data.keycloak_id).first()
if existing:
raise HTTPException(status_code=400, detail="User already exists")
user = User(**user_data.dict())
db.add(user)
db.commit()
db.refresh(user)
return user
@router.put("/{user_id}", response_model=UserSchema)
async def update_user(
user_id: int,
user_data: UserUpdateSchema,
db: Session = Depends(get_db),
current_user: dict = Depends(require_role(Roles.ADMIN))
):
"""Update user (Admin only)"""
user = db.query(User).filter(User.id == user_id).first()
if not user:
raise HTTPException(status_code=404, detail="User not found")
# Update fields
if user_data.email is not None:
user.email = user_data.email
if user_data.full_name is not None:
user.full_name = user_data.full_name
if user_data.is_active is not None:
user.is_active = user_data.is_active
# Update roles
if user_data.role_ids is not None:
roles = db.query(Role).filter(Role.id.in_(user_data.role_ids)).all()
user.roles = roles
db.commit()
db.refresh(user)
return user
@router.delete("/{user_id}")
async def delete_user(
user_id: int,
db: Session = Depends(get_db),
current_user: dict = Depends(require_role(Roles.ADMIN))
):
"""Delete user (Admin only)"""
user = db.query(User).filter(User.id == user_id).first()
if not user:
raise HTTPException(status_code=404, detail="User not found")
db.delete(user)
db.commit()
return {"message": "User deleted successfully"}
@router.get("/roles/", response_model=List[RoleSchema])
async def list_roles(
db: Session = Depends(get_db),
current_user: dict = Depends(require_role(Roles.ADMIN))
):
"""List all roles (Admin only)"""
roles = db.query(Role).all()
return roles

View File

@@ -0,0 +1,252 @@
"""
Authentication routes for Keycloak web login
Note: These routes are ONLY for web UI authentication
API endpoints use API Key authentication separately
"""
from fastapi import APIRouter, Request, HTTPException, Query
from fastapi.responses import RedirectResponse
from app.security.keycloak_auth import (
get_keycloak_client,
get_login_url,
get_logout_url,
get_current_user
)
from app.core.config import settings
import logging
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/auth", tags=["authentication"])
@router.get("/login")
async def login(request: Request, redirect_to: str = Query(default="/")):
"""
Redirect to Keycloak login page
Args:
redirect_to: Path to redirect after successful login
"""
# Check if already logged in
user = get_current_user(request)
if user:
return RedirectResponse(url=redirect_to)
# Generate Keycloak login URL
login_url = get_login_url(redirect_to)
return RedirectResponse(url=login_url)
@router.get("/callback")
async def auth_callback(
request: Request,
code: str = Query(...),
state: str = Query(default="/")
):
"""
Handle Keycloak callback after login
Args:
code: Authorization code from Keycloak
state: Original redirect path
"""
try:
if settings.DEBUG_AUTH:
logger.info("=" * 80)
logger.info("AUTHENTICATION CALLBACK RECEIVED")
logger.info(f"Authorization code: {code[:20]}...{code[-20:] if len(code) > 40 else code}")
logger.info(f"State (redirect_to): {state}")
logger.info(f"Request URL: {request.url}")
logger.info(f"Request headers: {dict(request.headers)}")
keycloak_client = get_keycloak_client()
if settings.DEBUG_AUTH:
logger.info("-" * 80)
logger.info("EXCHANGING CODE FOR TOKENS")
logger.info(f"Grant type: authorization_code")
logger.info(f"Code: {code[:20]}...")
logger.info(f"Redirect URI: {settings.KEYCLOAK_REDIRECT_URI}")
# Exchange authorization code for tokens
token_response = keycloak_client.token(
grant_type="authorization_code",
code=code,
redirect_uri=settings.KEYCLOAK_REDIRECT_URI
)
if settings.DEBUG_AUTH:
logger.info("TOKEN RESPONSE RECEIVED")
logger.info(f"Access token: {'*' * 20}...{token_response.get('access_token', '')[-20:] if token_response.get('access_token') else 'NONE'}")
logger.info(f"Refresh token: {'Present' if token_response.get('refresh_token') else 'None'}")
logger.info(f"Token type: {token_response.get('token_type', 'N/A')}")
logger.info(f"Expires in: {token_response.get('expires_in', 'N/A')} seconds")
# Get user information from token
access_token = token_response.get("access_token")
if not access_token:
logger.error("No access token in response!")
if settings.DEBUG_AUTH:
logger.error(f"Full token response: {token_response}")
raise HTTPException(
status_code=400,
detail="No access token received from Keycloak"
)
if settings.DEBUG_AUTH:
logger.info("-" * 80)
logger.info("FETCHING USER INFO")
from jose import jwt
userinfo = jwt.decode(access_token, key="", options={"verify_signature": False, "verify_aud": False})
logger.info(f"Decoded access_token: {userinfo.get('preferred_username')}")
# # 2. ดึง id_token ออกมา (ตัวนี้คือหัวใจของ OIDC)
# id_token = token_response.get("id_token")
# if not id_token:
# logger.error("No id_token in response!")
# if settings.DEBUG_AUTH:
# logger.error(f"Full token response: {token_response}")
# raise HTTPException(
# status_code=400,
# detail="No id_token received from Keycloak"
# )
# # 3. Decode id_token เพื่อเอาข้อมูล User (ไม่ต้องใช้ Key เพราะเราเชื่อถือ Connection นี้)
# userinfo = jwt.decode(
# id_token,
# key="",
# options={"verify_signature": False, "verify_aud": False}
# )
#userinfo = keycloak_client.userinfo(access_token)
# Extract roles from token
roles = []
# 1. Realm roles (roles ระดับ realm)
if "realm_access" in userinfo and "roles" in userinfo["realm_access"]:
roles.extend(userinfo["realm_access"]["roles"])
# 2. Client roles (roles เฉพาะ client apiservice)
if "resource_access" in userinfo and settings.KEYCLOAK_CLIENT_ID in userinfo["resource_access"]:
client_roles = userinfo["resource_access"][settings.KEYCLOAK_CLIENT_ID].get("roles", [])
roles.extend(client_roles)
# Filter to only include our application roles
user_roles = [r for r in roles if r in ["admin", "operation"]]
if settings.DEBUG_AUTH:
logger.info("USER INFO RECEIVED")
logger.info(f"Username: {userinfo.get('preferred_username')}")
logger.info(f"Email: {userinfo.get('email')}")
logger.info(f"Name: {userinfo.get('name')}")
logger.info(f"Sub (User ID): {userinfo.get('sub')}")
logger.info(f"All roles from token: {roles}")
logger.info(f"Filtered user roles: {user_roles}")
logger.info(f"Full userinfo keys: {list(userinfo.keys())}")
# Store user info, roles, and tokens in session
user_session_data = {
"username": userinfo.get("preferred_username"),
"email": userinfo.get("email"),
"name": userinfo.get("name", userinfo.get("preferred_username")),
"sub": userinfo.get("sub"), # User ID
"roles": user_roles, # User roles
"access_token": access_token,
"refresh_token": token_response.get("refresh_token")
}
request.session["user"] = user_session_data
if settings.DEBUG_AUTH:
logger.info("-" * 80)
logger.info("SESSION UPDATED")
logger.info(f"Session user data: {dict((k, v) for k, v in user_session_data.items() if k not in ['access_token', 'refresh_token'])}")
logger.info(f"User {userinfo.get('preferred_username')} logged in successfully")
# Redirect to original destination
redirect_url = state if state else "/"
# Ensure redirect URL starts with root_path if set
if settings.ROOT_PATH and not redirect_url.startswith(settings.ROOT_PATH):
redirect_url = f"{settings.ROOT_PATH}{redirect_url}"
if settings.DEBUG_AUTH:
logger.info(f"Redirecting to: {redirect_url}")
logger.info("=" * 80)
return RedirectResponse(url=redirect_url, status_code=302)
except Exception as e:
logger.error(f"Authentication callback failed: {e}")
if settings.DEBUG_AUTH:
import traceback
logger.error("FULL TRACEBACK:")
logger.error(traceback.format_exc())
logger.error("=" * 80)
raise HTTPException(
status_code=400,
detail=f"Authentication failed: {str(e)}"
)
@router.get("/logout")
async def logout(request: Request):
"""
Logout user and clear session
Redirects to Keycloak logout page
"""
user = get_current_user(request)
# Clear session
request.session.clear()
if user:
logger.info(f"User {user.get('username')} logged out")
# Get Keycloak logout URL
redirect_uri = f"{settings.ROOT_PATH}/" if settings.ROOT_PATH else "/"
logout_url = get_logout_url(redirect_uri)
return RedirectResponse(url=logout_url)
@router.get("/user")
async def get_user_info(request: Request):
"""
Get current authenticated user information
Returns 401 if not authenticated
"""
user = get_current_user(request)
if not user:
raise HTTPException(
status_code=401,
detail="Not authenticated"
)
# Return user info without sensitive tokens
return {
"username": user.get("username"),
"email": user.get("email"),
"name": user.get("name"),
"sub": user.get("sub")
}
@router.get("/status")
async def auth_status(request: Request):
"""
Check authentication status
Returns whether user is logged in
"""
user = get_current_user(request)
return {
"authenticated": user is not None,
"user": {
"username": user.get("username"),
"name": user.get("name")
} if user else None
}

View File

@@ -0,0 +1,330 @@
"""
Web page routes for the application
"""
import os
from datetime import datetime
from pathlib import Path
from typing import List
from fastapi import APIRouter, Request, UploadFile, File, Form, HTTPException, Depends
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates
from pydantic import BaseModel
from sqlalchemy.orm import Session
import asyncio
import logging
from app.core.config import settings
from app.security.permissions import require_role, Roles
from app.db.session import get_db
from app.models.upload import UploadHistory
from app.services.airflow_client import airflow_client
from app.services import minio_client
logger = logging.getLogger(__name__)
router = APIRouter()
# Setup templates
templates_dir = Path(__file__).parent.parent / "templates"
templates = Jinja2Templates(directory=str(templates_dir))
# Local fallback directory (used only if MinIO is not configured)
UPLOAD_DIR = Path("/data/uploads")
UPLOAD_DIR.mkdir(parents=True, exist_ok=True)
class UploadRecordSchema(BaseModel):
id: int
upload_id: str
filename: str
filepath: str
description: str | None = None
status: str
job_id: str | None = None
logs: str | None = None
uploaded_by: str | None = None
uploaded_at: datetime
updated_at: datetime | None = None
class Config:
from_attributes = True
@router.get("/", response_class=HTMLResponse)
async def index(request: Request):
"""Landing page with navigation menu"""
user = request.session.get("user")
return templates.TemplateResponse(
"index.html",
{
"request": request,
"root_path": settings.ROOT_PATH,
"user": user
}
)
@router.get("/data-management/finance", response_class=HTMLResponse)
async def finance_page(request: Request):
"""Finance Excel upload page - requires operation or admin role"""
user = request.session.get("user")
return templates.TemplateResponse(
"data_management_finance.html",
{
"request": request,
"root_path": settings.ROOT_PATH,
"user": user
}
)
@router.get("/admin/users", response_class=HTMLResponse)
async def admin_users_page(
request: Request,
current_user: dict = Depends(require_role(Roles.ADMIN))
):
"""User management page - Admin only"""
return templates.TemplateResponse(
"admin_users.html",
{
"request": request,
"root_path": settings.ROOT_PATH,
"user": current_user
}
)
@router.get("/api-management", response_class=HTMLResponse)
async def api_management_page(
request: Request,
current_user: dict = Depends(require_role(Roles.ADMIN))
):
"""API Key management page - Admin only"""
return templates.TemplateResponse(
"api_management.html",
{
"request": request,
"root_path": settings.ROOT_PATH,
"user": current_user
}
)
@router.post("/data-management/finance/upload")
async def upload_finance_file(
request: Request,
file: UploadFile = File(...),
description: str = Form(None),
db: Session = Depends(get_db)
):
"""
Handle finance Excel file upload
- Saves file to /data/uploads/
- Stores upload record in database
- Triggers Airflow job (to be implemented)
- Returns upload record
"""
# Validate file type
if not file.filename.endswith(('.xlsx', '.xls')):
raise HTTPException(
status_code=400,
detail="Invalid file type. Only .xlsx and .xls files are allowed."
)
# Generate unique filename
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
safe_filename = file.filename.replace(" ", "_")
unique_filename = f"{timestamp}_{safe_filename}"
# Read file content
try:
content = await file.read()
except Exception as e:
raise HTTPException(status_code=500, detail=f"Failed to read file: {str(e)}")
# Upload to MinIO finance bucket
object_key = f"finance/{unique_filename}"
try:
minio_client.upload_file(
bucket=settings.MINIO_BUCKET_FINANCE,
object_name=object_key,
data=content,
content_type="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
)
filepath_stored = object_key # store MinIO key in DB
except Exception as e:
logger.error(f"MinIO upload failed: {e}")
raise HTTPException(status_code=500, detail=f"Failed to upload file to storage: {str(e)}")
# Get username from session
user = request.session.get("user")
username = user.get("username") if user else "anonymous"
# Create upload record in database
upload_id = f"upload_{timestamp}"
upload_record = UploadHistory(
upload_id=upload_id,
filename=file.filename,
filepath=filepath_stored, # MinIO object key: finance/<filename>
description=description,
status="pending",
uploaded_by=username
)
db.add(upload_record)
db.commit()
db.refresh(upload_record)
# Trigger Airflow DAG with retry logic
airflow_triggered = False
dag_run_id = None
error_msg = None
max_retries = 3
retry_delay = 10 # seconds
for attempt in range(max_retries):
try:
logger.info(f"Triggering Airflow DAG (attempt {attempt + 1}/{max_retries})")
result = await airflow_client.trigger_finance_dag(
upload_id=upload_id,
filepath=str(filepath_stored),
filename=file.filename,
uploaded_by=username,
description=description
)
dag_run_id = result.get("dag_run_id")
airflow_triggered = True
# Update upload record with Airflow info
upload_record.airflow_dag_run_id = dag_run_id
upload_record.airflow_state = result.get("state", "queued")
upload_record.status = "processing"
db.commit()
logger.info(f"Airflow DAG triggered successfully: {dag_run_id}")
break
except Exception as e:
error_msg = str(e)
logger.error(f"Failed to trigger Airflow (attempt {attempt + 1}/{max_retries}): {error_msg}")
if attempt < max_retries - 1:
logger.info(f"Retrying in {retry_delay} seconds...")
await asyncio.sleep(retry_delay)
else:
logger.error(f"All {max_retries} attempts failed to trigger Airflow")
upload_record.status = "error"
upload_record.error_message = f"Failed to trigger Airflow after {max_retries} attempts: {error_msg}"
db.commit()
return {
"success": True,
"message": f"File '{file.filename}' uploaded successfully",
"upload_id": upload_id,
"filename": unique_filename,
"airflow_triggered": airflow_triggered,
"dag_run_id": dag_run_id,
"error": error_msg if not airflow_triggered else None
}
@router.get("/data-management/finance/uploads")
async def get_uploads(db: Session = Depends(get_db)):
"""Get list of all uploads with their status"""
uploads = db.query(UploadHistory).order_by(UploadHistory.uploaded_at.desc()).all()
# Convert to dict for JSON response
return [
{
"id": upload.upload_id,
"filename": upload.filename,
"filepath": upload.filepath,
"uploaded_at": upload.uploaded_at.isoformat(),
"description": upload.description,
"status": upload.status,
"job_id": upload.job_id,
"logs": upload.logs,
"uploaded_by": upload.uploaded_by,
"airflow_dag_run_id": upload.airflow_dag_run_id,
"airflow_state": upload.airflow_state,
"processing_started_at": upload.processing_started_at.isoformat() if upload.processing_started_at else None,
"processing_completed_at": upload.processing_completed_at.isoformat() if upload.processing_completed_at else None,
"error_message": upload.error_message
}
for upload in uploads
]
@router.get("/data-management/finance/uploads/{upload_id}")
async def get_upload_status(upload_id: str, db: Session = Depends(get_db)):
"""Get status of a specific upload"""
upload = db.query(UploadHistory).filter(UploadHistory.upload_id == upload_id).first()
if not upload:
raise HTTPException(status_code=404, detail="Upload not found")
return {
"id": upload.upload_id,
"filename": upload.filename,
"filepath": upload.filepath,
"uploaded_at": upload.uploaded_at.isoformat(),
"description": upload.description,
"status": upload.status,
"job_id": upload.job_id,
"logs": upload.logs,
"uploaded_by": upload.uploaded_by,
"airflow_dag_run_id": upload.airflow_dag_run_id,
"airflow_state": upload.airflow_state,
"processing_started_at": upload.processing_started_at.isoformat() if upload.processing_started_at else None,
"processing_completed_at": upload.processing_completed_at.isoformat() if upload.processing_completed_at else None,
"error_message": upload.error_message
}
# Placeholder for Airflow integration
async def trigger_airflow_job(filepath: str, upload_id: str) -> str:
"""
Trigger Airflow DAG to process the uploaded file
Args:
filepath: Path to the uploaded file
upload_id: Unique upload identifier
Returns:
job_id: Airflow job/run ID
This function will be implemented when:
- Airflow DAG ID is provided
- Airflow API endpoint is configured
"""
# TODO: Implement Airflow API call
# Example implementation:
# import httpx
#
# airflow_url = "http://airflow-webserver:8080/api/v1/dags/{dag_id}/dagRuns"
# headers = {"Content-Type": "application/json"}
# auth = ("airflow", "airflow") # Use proper credentials
#
# payload = {
# "conf": {
# "filepath": filepath,
# "upload_id": upload_id
# }
# }
#
# async with httpx.AsyncClient() as client:
# response = await client.post(
# airflow_url,
# json=payload,
# headers=headers,
# auth=auth
# )
# response.raise_for_status()
# result = response.json()
# return result["dag_run_id"]
raise NotImplementedError("Airflow integration pending DAG ID and endpoint")

View File

@@ -0,0 +1,146 @@
"""
Keycloak authentication for web pages
Note: This is ONLY for web UI authentication, NOT for API endpoints
API endpoints use API Key authentication (see app/security/dependencies.py)
"""
from typing import Optional
from fastapi import HTTPException, Request, status
from keycloak import KeycloakOpenID
from app.core.config import settings
import logging
logger = logging.getLogger(__name__)
# Initialize Keycloak OpenID client
def get_keycloak_client() -> KeycloakOpenID:
"""Get Keycloak OpenID client instance"""
try:
if settings.DEBUG_AUTH:
logger.info("=" * 80)
logger.info("KEYCLOAK CLIENT INITIALIZATION")
logger.info(f"Server URL: {settings.KEYCLOAK_SERVER_URL}")
logger.info(f"Realm: {settings.KEYCLOAK_REALM}")
logger.info(f"Client ID: {settings.KEYCLOAK_CLIENT_ID}")
logger.info(f"Client Secret: {'*' * len(settings.KEYCLOAK_CLIENT_SECRET) if settings.KEYCLOAK_CLIENT_SECRET else 'NOT SET'}")
logger.info(f"Redirect URI: {settings.KEYCLOAK_REDIRECT_URI}")
logger.info("=" * 80)
return KeycloakOpenID(
server_url=settings.KEYCLOAK_SERVER_URL,
client_id=settings.KEYCLOAK_CLIENT_ID,
realm_name=settings.KEYCLOAK_REALM,
client_secret_key=settings.KEYCLOAK_CLIENT_SECRET,
verify=True
)
except Exception as e:
logger.error(f"Failed to initialize Keycloak client: {e}")
if settings.DEBUG_AUTH:
import traceback
logger.error(f"Traceback: {traceback.format_exc()}")
raise
def get_current_user(request: Request) -> Optional[dict]:
"""
Get current authenticated user from session
Returns None if not authenticated
"""
return request.session.get("user")
def require_user(request: Request) -> dict:
"""
Dependency to require authenticated user
Raises 401 if not authenticated
"""
user = get_current_user(request)
if not user:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Authentication required"
)
return user
def get_login_url(redirect_to: str = "/") -> str:
"""
Generate Keycloak login URL
Args:
redirect_to: Path to redirect after successful login
Returns:
Keycloak authorization URL
"""
try:
if settings.DEBUG_AUTH:
logger.info("=" * 80)
logger.info("GENERATING LOGIN URL")
logger.info(f"Redirect to after login: {redirect_to}")
logger.info(f"Keycloak redirect URI: {settings.KEYCLOAK_REDIRECT_URI}")
keycloak_client = get_keycloak_client()
auth_url = keycloak_client.auth_url(
redirect_uri=settings.KEYCLOAK_REDIRECT_URI,
state=redirect_to
)
if settings.DEBUG_AUTH:
logger.info(f"Generated auth URL: {auth_url}")
logger.info("=" * 80)
return auth_url
except Exception as e:
logger.error(f"Failed to generate login URL: {e}")
if settings.DEBUG_AUTH:
import traceback
logger.error(f"Traceback: {traceback.format_exc()}")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Authentication service unavailable"
)
async def verify_token(token: str) -> dict:
"""
Verify and decode Keycloak access token
Args:
token: Access token from Keycloak
Returns:
User information from token
"""
try:
keycloak_client = get_keycloak_client()
userinfo = keycloak_client.userinfo(token)
return userinfo
except Exception as e:
logger.error(f"Token verification failed: {e}")
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid or expired token"
)
def get_logout_url(redirect_uri: str = None) -> str:
"""
Generate Keycloak logout URL
Args:
redirect_uri: Where to redirect after logout
Returns:
Keycloak logout URL
"""
try:
keycloak_client = get_keycloak_client()
if redirect_uri is None:
redirect_uri = f"{settings.ROOT_PATH}/" if settings.ROOT_PATH else "/"
logout_url = keycloak_client.logout_url(redirect_uri=redirect_uri)
return logout_url
except Exception as e:
logger.error(f"Failed to generate logout URL: {e}")
# Return simple redirect if Keycloak logout fails
return redirect_uri or "/"

View File

@@ -0,0 +1,151 @@
"""
Role-based permission system for web pages
Note: API endpoints (/api/v1/*) use API Key authentication and are not affected by this
"""
from typing import List
from fastapi import HTTPException, Request, status
from app.core.config import settings
import logging
logger = logging.getLogger(__name__)
class Roles:
"""Role constants"""
ADMIN = "admin"
OPERATION = "operation"
def get_user_roles(request: Request) -> List[str]:
"""
Get roles from session
Args:
request: FastAPI request object
Returns:
List of role names
"""
user = request.session.get("user")
if not user:
return []
return user.get("roles", [])
def has_role(request: Request, required_role: str) -> bool:
"""
Check if user has specific role
Args:
request: FastAPI request object
required_role: Role name to check
Returns:
True if user has the role, False otherwise
"""
user_roles = get_user_roles(request)
return required_role in user_roles
def has_any_role(request: Request, required_roles: List[str]) -> bool:
"""
Check if user has any of the required roles
Args:
request: FastAPI request object
required_roles: List of role names
Returns:
True if user has at least one of the roles, False otherwise
"""
user_roles = get_user_roles(request)
return any(role in user_roles for role in required_roles)
def require_role(required_role: str):
"""
Dependency to require specific role
Args:
required_role: Role name required to access the endpoint
Returns:
Dependency function that checks for the role
Raises:
HTTPException: 401 if not authenticated, 403 if missing required role
"""
def check_role(request: Request):
user = request.session.get("user")
if not user:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Authentication required"
)
user_roles = user.get("roles", [])
if required_role not in user_roles:
logger.warning(
f"Access denied: User {user.get('username')} "
f"(roles: {user_roles}) tried to access resource requiring '{required_role}'"
)
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail=f"Access denied. Required role: {required_role}"
)
return user
return check_role
def require_any_role(required_roles: List[str]):
"""
Dependency to require any of the specified roles
Args:
required_roles: List of role names, user needs at least one
Returns:
Dependency function that checks for roles
Raises:
HTTPException: 401 if not authenticated, 403 if missing all required roles
"""
def check_roles(request: Request):
user = request.session.get("user")
if not user:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Authentication required"
)
user_roles = user.get("roles", [])
if not any(role in user_roles for role in required_roles):
logger.warning(
f"Access denied: User {user.get('username')} "
f"(roles: {user_roles}) tried to access resource requiring one of: {required_roles}"
)
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail=f"Access denied. Required roles: {', '.join(required_roles)}"
)
return user
return check_roles
def is_admin(request: Request) -> bool:
"""
Check if user is admin
Args:
request: FastAPI request object
Returns:
True if user has admin role, False otherwise
"""
return has_role(request, Roles.ADMIN)

View File

@@ -0,0 +1,152 @@
"""
Airflow API Client for triggering DAGs
"""
import httpx
import logging
from typing import Dict, Any, Optional
from datetime import datetime
from app.core.config import settings
logger = logging.getLogger(__name__)
class AirflowClient:
"""Client for interacting with Airflow REST API"""
def __init__(self):
self.api_url = settings.AIRFLOW_API_URL.rstrip('/')
self.api_token = settings.AIRFLOW_API_TOKEN
self.headers = {
"Content-Type": "application/json",
"Accept": "application/json"
}
if self.api_token:
self.headers["Authorization"] = f"Bearer {self.api_token}"
else:
logger.warning("AIRFLOW_API_TOKEN not set - API calls may fail")
async def trigger_dag(
self,
dag_id: str,
conf: Dict[str, Any],
logical_date: Optional[str] = None
) -> Dict[str, Any]:
"""
Trigger an Airflow DAG run
Args:
dag_id: DAG identifier
conf: Configuration to pass to the DAG (JSON serializable dict)
logical_date: Optional logical date for the DAG run (ISO format)
Returns:
Dict containing dag_run_id and other metadata
Raises:
httpx.HTTPError: If the API call fails
"""
url = f"{self.api_url}/api/v1/dags/{dag_id}/dagRuns"
payload = {
"conf": conf
}
if logical_date:
payload["logical_date"] = logical_date
else:
payload["logical_date"] = datetime.utcnow().isoformat() + "Z"
logger.info(f"Triggering Airflow DAG: {dag_id}")
logger.debug(f"Payload: {payload}")
try:
async with httpx.AsyncClient(timeout=30.0) as client:
response = await client.post(
url,
json=payload,
headers=self.headers
)
response.raise_for_status()
result = response.json()
logger.info(f"DAG triggered successfully: {result.get('dag_run_id')}")
return result
except httpx.HTTPStatusError as e:
logger.error(f"HTTP error triggering DAG {dag_id}: {e.response.status_code} - {e.response.text}")
raise
except httpx.RequestError as e:
logger.error(f"Request error triggering DAG {dag_id}: {str(e)}")
raise
except Exception as e:
logger.error(f"Unexpected error triggering DAG {dag_id}: {str(e)}")
raise
async def get_dag_run_status(
self,
dag_id: str,
dag_run_id: str
) -> Dict[str, Any]:
"""
Get the status of a DAG run
Args:
dag_id: DAG identifier
dag_run_id: DAG run identifier
Returns:
Dict containing DAG run status and metadata
"""
url = f"{self.api_url}/api/v1/dags/{dag_id}/dagRuns/{dag_run_id}"
try:
async with httpx.AsyncClient(timeout=30.0) as client:
response = await client.get(url, headers=self.headers)
response.raise_for_status()
return response.json()
except httpx.HTTPError as e:
logger.error(f"Error getting DAG run status: {str(e)}")
raise
async def trigger_finance_dag(
self,
upload_id: str,
filepath: str,
filename: str,
uploaded_by: str,
description: Optional[str] = None
) -> Dict[str, Any]:
"""
Trigger the finance Excel processing DAG
Args:
upload_id: Unique upload identifier
filepath: Full path to the uploaded file
filename: Original filename
uploaded_by: Username who uploaded the file
description: Optional description
Returns:
Dict containing dag_run_id and other metadata
"""
conf = {
"upload_id": upload_id,
"filepath": filepath,
"filename": filename,
"uploaded_by": uploaded_by,
"description": description or "",
"triggered_at": datetime.utcnow().isoformat()
}
return await self.trigger_dag(
dag_id=settings.AIRFLOW_DAG_ID_FINANCE,
conf=conf
)
# Singleton instance
airflow_client = AirflowClient()

View File

@@ -0,0 +1,48 @@
import io
import logging
from minio import Minio
from minio.error import S3Error
from app.core.config import settings
logger = logging.getLogger(__name__)
_client: Minio | None = None
def get_client() -> Minio:
global _client
if _client is None:
_client = Minio(
endpoint=settings.MINIO_ENDPOINT,
access_key=settings.MINIO_ACCESS_KEY,
secret_key=settings.MINIO_SECRET_KEY,
secure=settings.MINIO_USE_SSL,
)
return _client
def upload_file(bucket: str, object_name: str, data: bytes, content_type: str = "application/octet-stream") -> str:
"""Upload bytes to MinIO. Returns the object key."""
client = get_client()
client.put_object(
bucket_name=bucket,
object_name=object_name,
data=io.BytesIO(data),
length=len(data),
content_type=content_type,
)
logger.info(f"Uploaded {object_name} to bucket {bucket}")
return object_name
def get_presigned_url(bucket: str, object_name: str, expires_seconds: int = 3600) -> str:
"""Generate presigned GET URL valid for expires_seconds (default 1h)."""
from datetime import timedelta
client = get_client()
return client.presigned_get_object(bucket, object_name, expires=timedelta(seconds=expires_seconds))
def delete_file(bucket: str, object_name: str) -> None:
client = get_client()
client.remove_object(bucket, object_name)
logger.info(f"Deleted {object_name} from bucket {bucket}")

View File

@@ -0,0 +1,360 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>User Management - Admin</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
min-height: 100vh;
padding: 20px;
}
.container {
max-width: 1400px;
margin: 0 auto;
background: white;
border-radius: 20px;
padding: 40px;
box-shadow: 0 20px 60px rgba(0,0,0,0.3);
}
.header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 30px;
padding-bottom: 20px;
border-bottom: 2px solid #e0e0e0;
}
h1 { color: #333; font-size: 32px; }
.user-info {
display: flex;
align-items: center;
gap: 15px;
}
.role-badge {
padding: 5px 15px;
border-radius: 20px;
font-size: 12px;
font-weight: bold;
text-transform: uppercase;
}
.role-admin {
background: linear-gradient(135deg, #ff6b6b 0%, #ee5a52 100%);
color: white;
}
.role-operation {
background: linear-gradient(135deg, #4ecdc4 0%, #44a08d 100%);
color: white;
}
.back-link {
display: inline-block;
margin-bottom: 20px;
color: #667eea;
text-decoration: none;
font-weight: 600;
font-size: 16px;
}
.back-link:hover { text-decoration: underline; }
.alert {
padding: 15px 20px;
margin-bottom: 20px;
border-radius: 8px;
font-weight: 500;
}
.alert-success {
background: #51cf66;
color: white;
}
.alert-error {
background: #ff6b6b;
color: white;
}
.stats {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
gap: 20px;
margin-bottom: 30px;
}
.stat-card {
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
color: white;
padding: 20px;
border-radius: 12px;
text-align: center;
}
.stat-number {
font-size: 36px;
font-weight: bold;
margin-bottom: 5px;
}
.stat-label {
font-size: 14px;
opacity: 0.9;
}
table {
width: 100%;
border-collapse: collapse;
margin-top: 20px;
background: white;
}
th, td {
padding: 15px;
text-align: left;
border-bottom: 1px solid #e0e0e0;
}
th {
background: #f8f9fa;
font-weight: 600;
color: #495057;
position: sticky;
top: 0;
}
tr:hover { background: #f8f9fa; }
.status-active {
color: #51cf66;
font-weight: 600;
}
.status-inactive {
color: #ff6b6b;
font-weight: 600;
}
.btn {
padding: 8px 16px;
border: none;
border-radius: 6px;
cursor: pointer;
font-size: 14px;
transition: all 0.3s;
font-weight: 500;
}
.btn-primary {
background: #667eea;
color: white;
}
.btn-primary:hover {
background: #5568d3;
transform: translateY(-2px);
}
.btn-danger {
background: #ff6b6b;
color: white;
}
.btn-danger:hover {
background: #ee5a52;
transform: translateY(-2px);
}
.loading {
text-align: center;
padding: 40px;
color: #666;
}
.empty-state {
text-align: center;
padding: 60px;
color: #999;
}
.empty-state-icon {
font-size: 64px;
margin-bottom: 20px;
}
</style>
</head>
<body>
<div class="container">
<a href="{{ root_path }}/" class="back-link">← Back to Dashboard</a>
<div class="header">
<h1>👥 User Management</h1>
{% if user %}
<div class="user-info">
<span>{{ user.name or user.username }}</span>
{% if user.roles %}
{% for role in user.roles %}
<span class="role-badge role-{{ role }}">{{ role }}</span>
{% endfor %}
{% endif %}
</div>
{% endif %}
</div>
<div id="alertContainer"></div>
<div class="stats" id="statsContainer">
<div class="stat-card">
<div class="stat-number" id="totalUsers">-</div>
<div class="stat-label">Total Users</div>
</div>
<div class="stat-card">
<div class="stat-number" id="activeUsers">-</div>
<div class="stat-label">Active Users</div>
</div>
<div class="stat-card">
<div class="stat-number" id="adminUsers">-</div>
<div class="stat-label">Admins</div>
</div>
<div class="stat-card">
<div class="stat-number" id="operationUsers">-</div>
<div class="stat-label">Operations</div>
</div>
</div>
<div id="tableContainer">
<div class="loading">Loading users...</div>
</div>
</div>
<script>
const rootPath = "{{ root_path }}";
async function loadUsers() {
try {
const response = await fetch(`${rootPath}/admin/users/`);
if (!response.ok) {
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
}
const users = await response.json();
// Update stats
document.getElementById('totalUsers').textContent = users.length;
document.getElementById('activeUsers').textContent = users.filter(u => u.is_active).length;
document.getElementById('adminUsers').textContent = users.filter(u =>
u.roles.some(r => r.name === 'admin')
).length;
document.getElementById('operationUsers').textContent = users.filter(u =>
u.roles.some(r => r.name === 'operation')
).length;
// Render table
const tableContainer = document.getElementById('tableContainer');
if (users.length === 0) {
tableContainer.innerHTML = `
<div class="empty-state">
<div class="empty-state-icon">📭</div>
<h3>No users found</h3>
<p>Users will appear here when they log in for the first time.</p>
</div>
`;
return;
}
tableContainer.innerHTML = `
<table>
<thead>
<tr>
<th>Username</th>
<th>Email</th>
<th>Full Name</th>
<th>Roles</th>
<th>Status</th>
<th>Last Login</th>
<th>Actions</th>
</tr>
</thead>
<tbody>
${users.map(user => `
<tr>
<td><strong>${escapeHtml(user.username)}</strong></td>
<td>${user.email ? escapeHtml(user.email) : '-'}</td>
<td>${user.full_name ? escapeHtml(user.full_name) : '-'}</td>
<td>
${user.roles.map(role =>
`<span class="role-badge role-${role.name}">${role.name}</span>`
).join(' ') || '<span style="color: #999;">No roles</span>'}
</td>
<td class="${user.is_active ? 'status-active' : 'status-inactive'}">
${user.is_active ? '✅ Active' : '❌ Inactive'}
</td>
<td>${user.last_login ? formatDate(user.last_login) : 'Never'}</td>
<td>
<button class="btn btn-primary" onclick="editUser(${user.id})">Edit</button>
<button class="btn btn-danger" onclick="deleteUser(${user.id}, '${escapeHtml(user.username)}')">Delete</button>
</td>
</tr>
`).join('')}
</tbody>
</table>
`;
} catch (error) {
console.error('Error loading users:', error);
document.getElementById('tableContainer').innerHTML = `
<div class="empty-state">
<div class="empty-state-icon">⚠️</div>
<h3>Error loading users</h3>
<p>${escapeHtml(error.message)}</p>
</div>
`;
showAlert('Failed to load users: ' + error.message, 'error');
}
}
function showAlert(message, type) {
const alertContainer = document.getElementById('alertContainer');
alertContainer.innerHTML = `
<div class="alert alert-${type}">
${escapeHtml(message)}
</div>
`;
setTimeout(() => alertContainer.innerHTML = '', 5000);
}
function editUser(userId) {
// TODO: Implement edit modal
showAlert('Edit functionality coming soon!', 'success');
}
async function deleteUser(userId, username) {
if (!confirm(`Are you sure you want to delete user "${username}"?`)) return;
try {
const response = await fetch(`${rootPath}/admin/users/${userId}`, {
method: 'DELETE'
});
if (response.ok) {
showAlert(`User "${username}" deleted successfully`, 'success');
loadUsers();
} else {
const error = await response.json();
showAlert('Failed to delete user: ' + error.detail, 'error');
}
} catch (error) {
console.error('Error deleting user:', error);
showAlert('Failed to delete user: ' + error.message, 'error');
}
}
function formatDate(dateString) {
const date = new Date(dateString);
return date.toLocaleString('en-US', {
year: 'numeric',
month: 'short',
day: 'numeric',
hour: '2-digit',
minute: '2-digit'
});
}
function escapeHtml(text) {
const div = document.createElement('div');
div.textContent = text;
return div.innerHTML;
}
// Load users on page load
loadUsers();
</script>
</body>
</html>

View File

@@ -0,0 +1,620 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>API Management - Admin</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
min-height: 100vh;
padding: 20px;
}
.container {
max-width: 1400px;
margin: 0 auto;
background: white;
border-radius: 20px;
padding: 40px;
box-shadow: 0 20px 60px rgba(0,0,0,0.3);
}
.header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 30px;
padding-bottom: 20px;
border-bottom: 2px solid #e0e0e0;
}
h1 { color: #333; font-size: 32px; }
.user-info { display: flex; align-items: center; gap: 15px; }
.role-badge {
padding: 5px 15px;
border-radius: 20px;
font-size: 12px;
font-weight: bold;
text-transform: uppercase;
background: linear-gradient(135deg, #ff6b6b 0%, #ee5a52 100%);
color: white;
}
.back-link {
display: inline-block;
margin-bottom: 20px;
color: #667eea;
text-decoration: none;
font-weight: 600;
font-size: 16px;
}
.back-link:hover { text-decoration: underline; }
.alert {
padding: 15px 20px;
margin-bottom: 20px;
border-radius: 8px;
font-weight: 500;
}
.alert-success { background: #51cf66; color: white; }
.alert-error { background: #ff6b6b; color: white; }
.stats {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(180px, 1fr));
gap: 20px;
margin-bottom: 30px;
}
.stat-card {
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
color: white;
padding: 20px;
border-radius: 12px;
text-align: center;
}
.stat-number { font-size: 36px; font-weight: bold; margin-bottom: 5px; }
.stat-label { font-size: 14px; opacity: 0.9; }
.section-header {
display: flex;
justify-content: space-between;
align-items: center;
margin: 30px 0 15px;
}
.section-header h2 { font-size: 22px; color: #333; }
.btn {
padding: 8px 16px;
border: none;
border-radius: 6px;
cursor: pointer;
font-size: 14px;
font-weight: 500;
transition: all 0.2s;
}
.btn-primary { background: #667eea; color: white; }
.btn-primary:hover { background: #5568d3; transform: translateY(-1px); }
.btn-success { background: #51cf66; color: white; }
.btn-success:hover { background: #40c057; transform: translateY(-1px); }
.btn-warning { background: #fcc419; color: #333; }
.btn-warning:hover { background: #fab005; transform: translateY(-1px); }
.btn-danger { background: #ff6b6b; color: white; }
.btn-danger:hover { background: #ee5a52; transform: translateY(-1px); }
.btn-sm { padding: 5px 10px; font-size: 12px; }
.client-card {
border: 1px solid #e0e0e0;
border-radius: 12px;
margin-bottom: 20px;
overflow: hidden;
}
.client-header {
display: flex;
justify-content: space-between;
align-items: center;
padding: 16px 20px;
background: #f8f9fa;
border-bottom: 1px solid #e0e0e0;
}
.client-name { font-size: 18px; font-weight: 600; color: #333; }
.client-meta { font-size: 13px; color: #666; margin-top: 3px; }
.client-actions { display: flex; gap: 8px; align-items: center; }
table { width: 100%; border-collapse: collapse; }
th, td { padding: 12px 16px; text-align: left; border-bottom: 1px solid #f0f0f0; }
th { font-size: 12px; font-weight: 600; color: #666; text-transform: uppercase; letter-spacing: 0.5px; background: #fafafa; }
tr:last-child td { border-bottom: none; }
tr:hover td { background: #f8f9ff; }
.status-active { color: #51cf66; font-weight: 600; }
.status-inactive { color: #ff6b6b; font-weight: 600; }
.key-prefix {
font-family: monospace;
background: #f1f3f5;
padding: 3px 8px;
border-radius: 4px;
font-size: 13px;
}
.perm-tag {
display: inline-block;
background: #e7f5ff;
color: #1c7ed6;
border-radius: 4px;
padding: 2px 8px;
font-size: 12px;
margin: 2px 2px 2px 0;
}
.empty-keys {
padding: 20px;
text-align: center;
color: #aaa;
font-size: 14px;
}
/* Modal */
.modal-overlay {
display: none;
position: fixed;
inset: 0;
background: rgba(0,0,0,0.5);
z-index: 1000;
align-items: center;
justify-content: center;
}
.modal-overlay.active { display: flex; }
.modal {
background: white;
border-radius: 16px;
padding: 32px;
max-width: 500px;
width: 90%;
box-shadow: 0 20px 60px rgba(0,0,0,0.3);
}
.modal h3 { font-size: 20px; margin-bottom: 20px; color: #333; }
.form-group { margin-bottom: 16px; }
.form-group label { display: block; font-size: 14px; font-weight: 600; color: #555; margin-bottom: 6px; }
.form-group input, .form-group textarea {
width: 100%;
padding: 10px 12px;
border: 1px solid #ddd;
border-radius: 8px;
font-size: 14px;
outline: none;
}
.form-group input:focus, .form-group textarea:focus { border-color: #667eea; }
.form-group small { font-size: 12px; color: #888; margin-top: 4px; display: block; }
.modal-actions { display: flex; justify-content: flex-end; gap: 10px; margin-top: 24px; }
.key-result-box {
background: #f1f3f5;
border: 1px solid #dee2e6;
border-radius: 8px;
padding: 12px 16px;
font-family: monospace;
font-size: 14px;
word-break: break-all;
margin: 10px 0;
}
.key-warning {
background: #fff3cd;
border: 1px solid #ffc107;
border-radius: 8px;
padding: 10px 14px;
font-size: 13px;
color: #856404;
margin-bottom: 12px;
}
.loading { text-align: center; padding: 40px; color: #666; }
.toggle-row { display: flex; align-items: center; gap: 10px; }
.toggle-switch {
position: relative; width: 44px; height: 24px; cursor: pointer;
}
.toggle-switch input { opacity: 0; width: 0; height: 0; }
.toggle-slider {
position: absolute; inset: 0; background: #ccc; border-radius: 24px; transition: 0.3s;
}
.toggle-slider:before {
content: ''; position: absolute; width: 18px; height: 18px;
left: 3px; top: 3px; background: white; border-radius: 50%; transition: 0.3s;
}
.toggle-switch input:checked + .toggle-slider { background: #51cf66; }
.toggle-switch input:checked + .toggle-slider:before { transform: translateX(20px); }
</style>
</head>
<body>
<div class="container">
<a href="{{ root_path }}/" class="back-link">← Back to Dashboard</a>
<div class="header">
<h1>🔑 API Management</h1>
{% if user %}
<div class="user-info">
<span>{{ user.name or user.username }}</span>
{% if user.roles %}
{% for role in user.roles %}
<span class="role-badge">{{ role }}</span>
{% endfor %}
{% endif %}
</div>
{% endif %}
</div>
<div id="alertContainer"></div>
<div class="stats">
<div class="stat-card">
<div class="stat-number" id="totalClients">-</div>
<div class="stat-label">API Clients</div>
</div>
<div class="stat-card">
<div class="stat-number" id="totalKeys">-</div>
<div class="stat-label">Total Keys</div>
</div>
<div class="stat-card">
<div class="stat-number" id="activeKeys">-</div>
<div class="stat-label">Active Keys</div>
</div>
</div>
<div class="section-header">
<h2>API Clients</h2>
<button class="btn btn-primary" onclick="openNewClientModal()">+ New Client</button>
</div>
<div id="clientsContainer">
<div class="loading">Loading...</div>
</div>
</div>
<!-- Modal: New Client -->
<div class="modal-overlay" id="newClientModal">
<div class="modal">
<h3>New API Client</h3>
<div class="form-group">
<label>Client Name</label>
<input type="text" id="newClientName" placeholder="e.g. hospital-erp" />
<small>Unique identifier for this client system</small>
</div>
<div class="modal-actions">
<button class="btn" onclick="closeModal('newClientModal')">Cancel</button>
<button class="btn btn-primary" onclick="createClient()">Create</button>
</div>
</div>
</div>
<!-- Modal: New Key -->
<div class="modal-overlay" id="newKeyModal">
<div class="modal">
<h3>Generate API Key</h3>
<input type="hidden" id="newKeyClientId" />
<div class="form-group">
<label>Key Name (optional)</label>
<input type="text" id="newKeyName" placeholder="e.g. production" />
</div>
<div class="form-group">
<label>Permissions</label>
<textarea id="newKeyPermissions" rows="4" placeholder='["voc.data:write", "feed.checkpoint:write"]'></textarea>
<small>JSON array of permission strings</small>
</div>
<div class="modal-actions">
<button class="btn" onclick="closeModal('newKeyModal')">Cancel</button>
<button class="btn btn-success" onclick="generateKey()">Generate</button>
</div>
</div>
</div>
<!-- Modal: Edit Client -->
<div class="modal-overlay" id="editClientModal">
<div class="modal">
<h3>Edit API Client</h3>
<input type="hidden" id="editClientId" />
<div class="form-group">
<label>Client Name</label>
<input type="text" id="editClientName" />
</div>
<div class="form-group">
<label>Status</label>
<div class="toggle-row">
<label class="toggle-switch">
<input type="checkbox" id="editClientActive" onchange="updateClientActiveLabel()" />
<span class="toggle-slider"></span>
</label>
<span id="editClientActiveLabel">Active</span>
</div>
</div>
<div class="modal-actions">
<button class="btn" onclick="closeModal('editClientModal')">Cancel</button>
<button class="btn btn-primary" onclick="saveEditClient()">Save</button>
</div>
</div>
</div>
<!-- Modal: Edit Key -->
<div class="modal-overlay" id="editKeyModal">
<div class="modal">
<h3>Edit API Key</h3>
<input type="hidden" id="editKeyId" />
<div class="form-group">
<label>Key Name (optional)</label>
<input type="text" id="editKeyName" placeholder="e.g. production" />
</div>
<div class="form-group">
<label>Permissions</label>
<textarea id="editKeyPermissions" rows="4" placeholder='["voc.data:write"]'></textarea>
<small>JSON array of permission strings</small>
</div>
<div class="modal-actions">
<button class="btn" onclick="closeModal('editKeyModal')">Cancel</button>
<button class="btn btn-primary" onclick="saveEditKey()">Save</button>
</div>
</div>
</div>
<!-- Modal: Show Key -->
<div class="modal-overlay" id="keyResultModal">
<div class="modal">
<h3>🔑 API Key Generated</h3>
<div class="key-warning">⚠️ Copy this key now — it will NOT be shown again after closing this dialog.</div>
<div class="form-group">
<label>API Key</label>
<div class="key-result-box" id="keyResultValue"></div>
<button class="btn btn-primary btn-sm" onclick="copyKey()" style="margin-top:8px;">Copy to Clipboard</button>
</div>
<div class="form-group">
<label>Permissions</label>
<div id="keyResultPerms"></div>
</div>
<div class="modal-actions">
<button class="btn btn-primary" onclick="closeKeyResult()">Done</button>
</div>
</div>
</div>
<script>
const rootPath = "{{ root_path }}";
async function loadClients() {
try {
const res = await fetch(`${rootPath}/admin/api-keys/clients`);
if (!res.ok) throw new Error(`HTTP ${res.status}`);
const clients = await res.json();
const totalKeys = clients.reduce((s, c) => s + c.api_keys.length, 0);
const activeKeys = clients.reduce((s, c) => s + c.api_keys.filter(k => k.is_active).length, 0);
document.getElementById('totalClients').textContent = clients.length;
document.getElementById('totalKeys').textContent = totalKeys;
document.getElementById('activeKeys').textContent = activeKeys;
const container = document.getElementById('clientsContainer');
if (clients.length === 0) {
container.innerHTML = '<div class="empty-keys">No API clients yet. Create one to get started.</div>';
return;
}
container.innerHTML = clients.map(client => `
<div class="client-card">
<div class="client-header">
<div>
<div class="client-name">${escapeHtml(client.name)}</div>
<div class="client-meta">ID: ${client.id} &nbsp;·&nbsp; ${client.api_keys.length} key(s)</div>
</div>
<div class="client-actions">
<span class="${client.is_active ? 'status-active' : 'status-inactive'}">${client.is_active ? '● Active' : '● Inactive'}</span>
<button class="btn btn-success btn-sm" onclick="openNewKeyModal(${client.id})">+ Add Key</button>
<button class="btn btn-warning btn-sm" data-id="${client.id}" data-name="${escapeHtml(client.name)}" data-active="${client.is_active}" onclick="openEditClientModal(this)">Edit</button>
</div>
</div>
${client.api_keys.length === 0
? '<div class="empty-keys">No API keys yet.</div>'
: `<table>
<thead><tr>
<th>ID</th><th>Name</th><th>Prefix</th><th>Permissions</th><th>Status</th><th>Created</th><th>Actions</th>
</tr></thead>
<tbody>
${client.api_keys.map(key => `
<tr>
<td>${key.id}</td>
<td>${key.name ? escapeHtml(key.name) : '-'}</td>
<td><span class="key-prefix">${escapeHtml(key.key_prefix)}...</span></td>
<td>${key.permissions.map(p => `<span class="perm-tag">${escapeHtml(p)}</span>`).join('') || '<span style="color:#aaa">none</span>'}</td>
<td class="${key.is_active ? 'status-active' : 'status-inactive'}">${key.is_active ? '● Active' : '● Inactive'}</td>
<td>${formatDate(key.created_at)}</td>
<td>
<button class="btn btn-primary btn-sm" data-id="${key.id}" data-name="${escapeHtml(key.name || '')}" data-perms="${JSON.stringify(key.permissions).replace(/"/g, '&quot;')}" onclick="openEditKeyModal(this)">Edit</button>
<button class="btn btn-warning btn-sm" onclick="regenerateKey(${key.id})">Regenerate</button>
<button class="btn btn-sm" style="background:#dee2e6" onclick="toggleKey(${key.id})">${key.is_active ? 'Deactivate' : 'Activate'}</button>
</td>
</tr>
`).join('')}
</tbody>
</table>`
}
</div>
`).join('');
} catch (err) {
document.getElementById('clientsContainer').innerHTML = `<div class="empty-keys" style="color:#ff6b6b">Error: ${escapeHtml(err.message)}</div>`;
showAlert('Failed to load clients: ' + err.message, 'error');
}
}
function updateClientActiveLabel() {
const cb = document.getElementById('editClientActive');
document.getElementById('editClientActiveLabel').textContent = cb.checked ? 'Active' : 'Inactive';
}
function openEditClientModal(btn) {
const clientId = btn.dataset.id;
const name = btn.dataset.name;
const isActive = btn.dataset.active === 'true';
document.getElementById('editClientId').value = clientId;
document.getElementById('editClientName').value = name;
document.getElementById('editClientActive').checked = isActive;
document.getElementById('editClientActiveLabel').textContent = isActive ? 'Active' : 'Inactive';
document.getElementById('editClientModal').classList.add('active');
}
async function saveEditClient() {
const clientId = document.getElementById('editClientId').value;
const name = document.getElementById('editClientName').value.trim();
const isActive = document.getElementById('editClientActive').checked;
if (!name) return showAlert('Client name is required', 'error');
try {
const res = await fetch(`${rootPath}/admin/api-keys/clients/${clientId}`, {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ name, is_active: isActive })
});
if (!res.ok) { const e = await res.json(); throw new Error(e.detail || res.statusText); }
closeModal('editClientModal');
showAlert('Client updated', 'success');
loadClients();
} catch (err) { showAlert('Failed: ' + err.message, 'error'); }
}
function openEditKeyModal(btn) {
const keyId = btn.dataset.id;
const name = btn.dataset.name;
let permissions = [];
try { permissions = JSON.parse(btn.dataset.perms); } catch {}
document.getElementById('editKeyId').value = keyId;
document.getElementById('editKeyName').value = name || '';
document.getElementById('editKeyPermissions').value = JSON.stringify(permissions, null, 2);
document.getElementById('editKeyModal').classList.add('active');
}
async function saveEditKey() {
const keyId = document.getElementById('editKeyId').value;
const name = document.getElementById('editKeyName').value.trim() || null;
const permsRaw = document.getElementById('editKeyPermissions').value.trim();
let permissions;
try { permissions = permsRaw ? JSON.parse(permsRaw) : []; }
catch { return showAlert('Permissions must be a valid JSON array', 'error'); }
try {
const res = await fetch(`${rootPath}/admin/api-keys/${keyId}`, {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ name, permissions })
});
if (!res.ok) { const e = await res.json(); throw new Error(e.detail || res.statusText); }
closeModal('editKeyModal');
showAlert('Key updated', 'success');
loadClients();
} catch (err) { showAlert('Failed: ' + err.message, 'error'); }
}
function openNewClientModal() {
document.getElementById('newClientName').value = '';
document.getElementById('newClientModal').classList.add('active');
}
async function createClient() {
const name = document.getElementById('newClientName').value.trim();
if (!name) return showAlert('Client name is required', 'error');
try {
const res = await fetch(`${rootPath}/admin/api-keys/clients`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ name })
});
if (!res.ok) { const e = await res.json(); throw new Error(e.detail || res.statusText); }
closeModal('newClientModal');
showAlert(`Client "${name}" created`, 'success');
loadClients();
} catch (err) { showAlert('Failed: ' + err.message, 'error'); }
}
function openNewKeyModal(clientId) {
document.getElementById('newKeyClientId').value = clientId;
document.getElementById('newKeyName').value = '';
document.getElementById('newKeyPermissions').value = '';
document.getElementById('newKeyModal').classList.add('active');
}
async function generateKey() {
const clientId = parseInt(document.getElementById('newKeyClientId').value);
const name = document.getElementById('newKeyName').value.trim() || null;
const permsRaw = document.getElementById('newKeyPermissions').value.trim();
let permissions = [];
if (permsRaw) {
try { permissions = JSON.parse(permsRaw); }
catch { return showAlert('Permissions must be a valid JSON array', 'error'); }
}
try {
const res = await fetch(`${rootPath}/admin/api-keys/generate`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ client_id: clientId, name, permissions })
});
if (!res.ok) { const e = await res.json(); throw new Error(e.detail || res.statusText); }
const data = await res.json();
closeModal('newKeyModal');
showKeyResult(data.api_key, data.permissions);
loadClients();
} catch (err) { showAlert('Failed: ' + err.message, 'error'); }
}
async function regenerateKey(keyId) {
if (!confirm('Regenerate this key? The current key will stop working immediately.')) return;
try {
const res = await fetch(`${rootPath}/admin/api-keys/${keyId}/regenerate`, { method: 'POST' });
if (!res.ok) { const e = await res.json(); throw new Error(e.detail || res.statusText); }
const data = await res.json();
showKeyResult(data.api_key, data.permissions);
loadClients();
} catch (err) { showAlert('Failed: ' + err.message, 'error'); }
}
async function toggleKey(keyId) {
try {
const res = await fetch(`${rootPath}/admin/api-keys/${keyId}/toggle`, { method: 'PATCH' });
if (!res.ok) { const e = await res.json(); throw new Error(e.detail || res.statusText); }
const data = await res.json();
showAlert(`Key ${data.is_active ? 'activated' : 'deactivated'}`, 'success');
loadClients();
} catch (err) { showAlert('Failed: ' + err.message, 'error'); }
}
function showKeyResult(apiKey, permissions) {
document.getElementById('keyResultValue').textContent = apiKey;
document.getElementById('keyResultPerms').innerHTML = permissions.length
? permissions.map(p => `<span class="perm-tag">${escapeHtml(p)}</span>`).join('')
: '<span style="color:#aaa">none</span>';
document.getElementById('keyResultModal').classList.add('active');
}
function copyKey() {
const key = document.getElementById('keyResultValue').textContent;
navigator.clipboard.writeText(key).then(() => showAlert('Copied to clipboard', 'success'));
}
function closeKeyResult() {
closeModal('keyResultModal');
}
function closeModal(id) {
document.getElementById(id).classList.remove('active');
}
function showAlert(message, type) {
const el = document.getElementById('alertContainer');
el.innerHTML = `<div class="alert alert-${type}">${escapeHtml(message)}</div>`;
setTimeout(() => el.innerHTML = '', 5000);
}
function formatDate(s) {
return new Date(s).toLocaleString('th-TH', { year: 'numeric', month: 'short', day: 'numeric', hour: '2-digit', minute: '2-digit' });
}
function escapeHtml(text) {
const d = document.createElement('div');
d.textContent = String(text);
return d.innerHTML;
}
loadClients();
</script>
</body>
</html>

View File

@@ -0,0 +1,635 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Finance Excel Upload - Sriphat Data Platform</title>
<style>
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
min-height: 100vh;
padding: 20px;
}
.container {
max-width: 1000px;
margin: 0 auto;
}
.header {
background: white;
border-radius: 12px;
padding: 25px;
margin-bottom: 20px;
box-shadow: 0 4px 6px rgba(0,0,0,0.1);
position: relative;
}
.header h1 {
color: #333;
margin-bottom: 10px;
display: flex;
align-items: center;
gap: 10px;
}
.header p {
color: #666;
}
.user-info {
position: absolute;
top: 25px;
right: 25px;
display: flex;
align-items: center;
gap: 15px;
background: #f8f9fa;
padding: 8px 16px;
border-radius: 20px;
border: 1px solid #dee2e6;
}
.user-info .username {
font-size: 0.9rem;
color: #495057;
font-weight: 500;
}
.logout-btn {
background: #667eea;
color: white;
padding: 5px 12px;
border-radius: 12px;
text-decoration: none;
font-size: 0.85rem;
transition: background 0.3s ease;
}
.logout-btn:hover {
background: #5568d3;
}
.role-badge {
padding: 4px 12px;
border-radius: 12px;
font-size: 11px;
font-weight: bold;
text-transform: uppercase;
letter-spacing: 0.5px;
}
.role-admin {
background: linear-gradient(135deg, #ff6b6b 0%, #ee5a52 100%);
color: white;
box-shadow: 0 2px 4px rgba(255,107,107,0.3);
}
.role-operation {
background: linear-gradient(135deg, #4ecdc4 0%, #44a08d 100%);
color: white;
box-shadow: 0 2px 4px rgba(78,205,196,0.3);
}
.back-link {
display: inline-block;
color: #667eea;
text-decoration: none;
margin-bottom: 15px;
font-size: 0.9rem;
}
.back-link:hover {
text-decoration: underline;
}
.upload-section {
background: white;
border-radius: 12px;
padding: 30px;
margin-bottom: 20px;
box-shadow: 0 4px 6px rgba(0,0,0,0.1);
}
.upload-section h2 {
color: #333;
margin-bottom: 20px;
font-size: 1.5rem;
}
.upload-form {
display: flex;
flex-direction: column;
gap: 20px;
}
.form-group {
display: flex;
flex-direction: column;
gap: 8px;
}
.form-group label {
color: #333;
font-weight: 500;
}
.file-input-wrapper {
position: relative;
display: inline-block;
width: 100%;
}
.file-input {
display: none;
}
.file-input-label {
display: flex;
align-items: center;
justify-content: center;
padding: 40px;
border: 2px dashed #667eea;
border-radius: 8px;
background: #f8f9fa;
cursor: pointer;
transition: all 0.3s ease;
}
.file-input-label:hover {
background: #e9ecef;
border-color: #764ba2;
}
.file-input-label.has-file {
border-color: #28a745;
background: #d4edda;
}
.file-info {
display: none;
margin-top: 10px;
padding: 10px;
background: #e9ecef;
border-radius: 6px;
font-size: 0.9rem;
}
.file-info.show {
display: block;
}
.btn {
padding: 12px 30px;
border: none;
border-radius: 6px;
font-size: 1rem;
cursor: pointer;
transition: all 0.3s ease;
font-weight: 500;
}
.btn-primary {
background: #667eea;
color: white;
}
.btn-primary:hover {
background: #5568d3;
}
.btn-primary:disabled {
background: #ccc;
cursor: not-allowed;
}
.alert {
padding: 15px;
border-radius: 6px;
margin-bottom: 20px;
display: none;
}
.alert.show {
display: block;
}
.alert-success {
background: #d4edda;
color: #155724;
border: 1px solid #c3e6cb;
}
.alert-error {
background: #f8d7da;
color: #721c24;
border: 1px solid #f5c6cb;
}
.alert-info {
background: #d1ecf1;
color: #0c5460;
border: 1px solid #bee5eb;
}
.uploads-list {
background: white;
border-radius: 12px;
padding: 30px;
box-shadow: 0 4px 6px rgba(0,0,0,0.1);
}
.uploads-list h2 {
color: #333;
margin-bottom: 20px;
font-size: 1.5rem;
}
.upload-item {
border: 1px solid #dee2e6;
border-radius: 8px;
margin-bottom: 15px;
overflow: hidden;
}
.upload-header {
padding: 15px;
background: #f8f9fa;
cursor: pointer;
display: flex;
justify-content: space-between;
align-items: center;
transition: background 0.2s ease;
}
.upload-header:hover {
background: #e9ecef;
}
.upload-info {
flex: 1;
}
.upload-filename {
font-weight: 500;
color: #333;
margin-bottom: 5px;
}
.upload-meta {
font-size: 0.85rem;
color: #666;
}
.upload-status {
display: flex;
align-items: center;
gap: 10px;
}
.status-badge {
padding: 5px 12px;
border-radius: 20px;
font-size: 0.85rem;
font-weight: 500;
}
.status-pending {
background: #fff3cd;
color: #856404;
}
.status-processing {
background: #d1ecf1;
color: #0c5460;
}
.status-success {
background: #d4edda;
color: #155724;
}
.status-error {
background: #f8d7da;
color: #721c24;
}
.expand-icon {
transition: transform 0.3s ease;
}
.expand-icon.expanded {
transform: rotate(180deg);
}
.upload-details {
padding: 15px;
background: white;
border-top: 1px solid #dee2e6;
display: none;
}
.upload-details.show {
display: block;
}
.log-container {
background: #f8f9fa;
border: 1px solid #dee2e6;
border-radius: 6px;
padding: 15px;
font-family: 'Courier New', monospace;
font-size: 0.85rem;
max-height: 300px;
overflow-y: auto;
white-space: pre-wrap;
word-wrap: break-word;
}
.log-error {
color: #dc3545;
}
.log-info {
color: #17a2b8;
}
.empty-state {
text-align: center;
padding: 40px;
color: #666;
}
.empty-state .icon {
font-size: 3rem;
margin-bottom: 15px;
opacity: 0.5;
}
.spinner {
display: inline-block;
width: 20px;
height: 20px;
border: 3px solid rgba(0,0,0,0.1);
border-radius: 50%;
border-top-color: #667eea;
animation: spin 1s ease-in-out infinite;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
@media (max-width: 768px) {
.container {
padding: 10px;
}
.upload-section, .uploads-list {
padding: 20px;
}
}
</style>
</head>
<body>
<div class="container">
<a href="{{ root_path }}/" class="back-link">← Back to Dashboard</a>
<div class="header">
<h1>💰 Finance Excel Upload</h1>
<p>Upload Excel files for financial data processing</p>
{% if user %}
<div class="user-info">
<span class="username">👤 {{ user.name or user.username }}</span>
{% if user.roles %}
{% for role in user.roles %}
<span class="role-badge role-{{ role }}">{{ role }}</span>
{% endfor %}
{% endif %}
<a href="{{ root_path }}/auth/logout" class="logout-btn">Logout</a>
</div>
{% endif %}
</div>
<div id="alertContainer"></div>
<div class="upload-section">
<h2>📤 Upload File</h2>
<form id="uploadForm" class="upload-form" enctype="multipart/form-data">
<div class="form-group">
<label for="file">Select Excel File (.xlsx, .xls)</label>
<div class="file-input-wrapper">
<input type="file" id="file" name="file" class="file-input" accept=".xlsx,.xls" required>
<label for="file" id="fileLabel" class="file-input-label">
<span>📁 Click to select file or drag and drop</span>
</label>
</div>
<div id="fileInfo" class="file-info"></div>
</div>
<div class="form-group">
<label for="description">Description (Optional)</label>
<textarea id="description" name="description" rows="3" style="padding: 10px; border: 1px solid #dee2e6; border-radius: 6px; font-family: inherit;"></textarea>
</div>
<button type="submit" class="btn btn-primary" id="submitBtn">
Upload and Process
</button>
</form>
</div>
<div class="uploads-list">
<h2>📋 Upload History</h2>
<div id="uploadsList">
<div class="empty-state">
<div class="icon">📭</div>
<p>No uploads yet</p>
</div>
</div>
</div>
</div>
<script>
const rootPath = '{{ root_path }}';
const fileInput = document.getElementById('file');
const fileLabel = document.getElementById('fileLabel');
const fileInfo = document.getElementById('fileInfo');
const uploadForm = document.getElementById('uploadForm');
const submitBtn = document.getElementById('submitBtn');
const uploadsList = document.getElementById('uploadsList');
const alertContainer = document.getElementById('alertContainer');
// File input handling
fileInput.addEventListener('change', function(e) {
const file = e.target.files[0];
if (file) {
fileLabel.classList.add('has-file');
fileLabel.innerHTML = `<span>✅ ${file.name}</span>`;
fileInfo.classList.add('show');
fileInfo.textContent = `File: ${file.name} (${formatFileSize(file.size)})`;
}
});
// Drag and drop
fileLabel.addEventListener('dragover', (e) => {
e.preventDefault();
fileLabel.style.borderColor = '#764ba2';
});
fileLabel.addEventListener('dragleave', () => {
fileLabel.style.borderColor = '#667eea';
});
fileLabel.addEventListener('drop', (e) => {
e.preventDefault();
fileLabel.style.borderColor = '#667eea';
const files = e.dataTransfer.files;
if (files.length > 0) {
fileInput.files = files;
fileInput.dispatchEvent(new Event('change'));
}
});
// Form submission
uploadForm.addEventListener('submit', async function(e) {
e.preventDefault();
const formData = new FormData(uploadForm);
submitBtn.disabled = true;
submitBtn.innerHTML = '<span class="spinner"></span> Uploading...';
try {
const response = await fetch(`${rootPath}/data-management/finance/upload`, {
method: 'POST',
body: formData
});
const result = await response.json();
if (response.ok) {
showAlert('success', result.message || 'File uploaded successfully!');
uploadForm.reset();
fileLabel.classList.remove('has-file');
fileLabel.innerHTML = '<span>📁 Click to select file or drag and drop</span>';
fileInfo.classList.remove('show');
// Refresh uploads list
loadUploads();
} else {
showAlert('error', result.detail || 'Upload failed');
}
} catch (error) {
showAlert('error', 'Network error: ' + error.message);
} finally {
submitBtn.disabled = false;
submitBtn.textContent = 'Upload and Process';
}
});
// Load uploads list
async function loadUploads() {
try {
const response = await fetch(`${rootPath}/data-management/finance/uploads`);
const uploads = await response.json();
if (uploads.length === 0) {
uploadsList.innerHTML = `
<div class="empty-state">
<div class="icon">📭</div>
<p>No uploads yet</p>
</div>
`;
return;
}
uploadsList.innerHTML = uploads.map(upload => `
<div class="upload-item">
<div class="upload-header" onclick="toggleDetails('${upload.id}')">
<div class="upload-info">
<div class="upload-filename">${upload.filename}</div>
<div class="upload-meta">
Uploaded: ${formatDate(upload.uploaded_at)}
${upload.description ? `${upload.description}` : ''}
</div>
</div>
<div class="upload-status">
<span class="status-badge status-${upload.status}">${upload.status.toUpperCase()}</span>
<span class="expand-icon" id="icon-${upload.id}">▼</span>
</div>
</div>
<div class="upload-details" id="details-${upload.id}">
${upload.job_id ? `<p><strong>Job ID:</strong> ${upload.job_id}</p>` : ''}
${upload.logs ? `
<h4>Logs:</h4>
<div class="log-container ${upload.status === 'error' ? 'log-error' : 'log-info'}">
${upload.logs}
</div>
` : '<p>No logs available</p>'}
</div>
</div>
`).join('');
} catch (error) {
console.error('Failed to load uploads:', error);
}
}
function toggleDetails(id) {
const details = document.getElementById(`details-${id}`);
const icon = document.getElementById(`icon-${id}`);
if (details.classList.contains('show')) {
details.classList.remove('show');
icon.classList.remove('expanded');
} else {
details.classList.add('show');
icon.classList.add('expanded');
}
}
function showAlert(type, message) {
const alertClass = type === 'success' ? 'alert-success' : type === 'error' ? 'alert-error' : 'alert-info';
const alert = document.createElement('div');
alert.className = `alert ${alertClass} show`;
alert.textContent = message;
alertContainer.innerHTML = '';
alertContainer.appendChild(alert);
setTimeout(() => {
alert.classList.remove('show');
}, 5000);
}
function formatFileSize(bytes) {
if (bytes === 0) return '0 Bytes';
const k = 1024;
const sizes = ['Bytes', 'KB', 'MB', 'GB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return Math.round(bytes / Math.pow(k, i) * 100) / 100 + ' ' + sizes[i];
}
function formatDate(dateString) {
const date = new Date(dateString);
return date.toLocaleString('en-US', {
year: 'numeric',
month: 'short',
day: 'numeric',
hour: '2-digit',
minute: '2-digit'
});
}
// Load uploads on page load
loadUploads();
// Refresh uploads every 10 seconds
setInterval(loadUploads, 10000);
</script>
</body>
</html>

View File

@@ -0,0 +1,305 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Sriphat Data Platform</title>
<style>
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
min-height: 100vh;
padding: 20px;
}
.container {
max-width: 1200px;
margin: 0 auto;
}
.header {
text-align: center;
color: white;
margin-bottom: 40px;
padding: 20px;
position: relative;
}
.header h1 {
font-size: 2.5rem;
margin-bottom: 10px;
text-shadow: 2px 2px 4px rgba(0,0,0,0.3);
}
.header p {
font-size: 1.1rem;
opacity: 0.9;
}
.user-info {
position: absolute;
top: 20px;
right: 20px;
display: flex;
align-items: center;
gap: 15px;
background: rgba(255,255,255,0.1);
padding: 10px 20px;
border-radius: 25px;
backdrop-filter: blur(10px);
}
.user-info .username {
font-size: 0.95rem;
font-weight: 500;
}
.logout-btn {
background: rgba(255,255,255,0.2);
color: white;
padding: 6px 15px;
border-radius: 15px;
text-decoration: none;
font-size: 0.9rem;
transition: background 0.3s ease;
}
.logout-btn:hover {
background: rgba(255,255,255,0.3);
}
.role-badge {
padding: 4px 12px;
border-radius: 12px;
font-size: 11px;
font-weight: bold;
text-transform: uppercase;
letter-spacing: 0.5px;
}
.role-admin {
background: linear-gradient(135deg, #ff6b6b 0%, #ee5a52 100%);
color: white;
box-shadow: 0 2px 4px rgba(255,107,107,0.3);
}
.role-operation {
background: linear-gradient(135deg, #4ecdc4 0%, #44a08d 100%);
color: white;
box-shadow: 0 2px 4px rgba(78,205,196,0.3);
}
.menu-grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(280px, 1fr));
gap: 20px;
margin-bottom: 30px;
}
.menu-card {
background: white;
border-radius: 12px;
padding: 25px;
box-shadow: 0 4px 6px rgba(0,0,0,0.1);
transition: transform 0.3s ease, box-shadow 0.3s ease;
cursor: pointer;
text-decoration: none;
color: inherit;
display: block;
}
.menu-card:hover {
transform: translateY(-5px);
box-shadow: 0 8px 15px rgba(0,0,0,0.2);
}
.menu-card.has-submenu {
cursor: default;
}
.menu-card .icon {
font-size: 2.5rem;
margin-bottom: 15px;
display: block;
}
.menu-card h3 {
color: #333;
margin-bottom: 10px;
font-size: 1.3rem;
}
.menu-card p {
color: #666;
font-size: 0.9rem;
line-height: 1.5;
}
.submenu {
margin-top: 15px;
padding-top: 15px;
border-top: 1px solid #eee;
display: none;
}
.submenu.active {
display: block;
}
.submenu-item {
display: block;
padding: 10px 15px;
margin: 5px 0;
background: #f8f9fa;
border-radius: 6px;
color: #495057;
text-decoration: none;
transition: background 0.2s ease;
}
.submenu-item:hover {
background: #e9ecef;
}
.submenu-toggle {
display: inline-block;
margin-left: 10px;
font-size: 0.8rem;
color: #667eea;
cursor: pointer;
}
/* Color themes for different services */
.card-supabase { border-left: 4px solid #3ECF8E; }
.card-api { border-left: 4px solid #009688; }
.card-airflow { border-left: 4px solid #017CEE; }
.card-airbyte { border-left: 4px solid #615EFF; }
.card-data { border-left: 4px solid #FF6B6B; }
.card-dbt { border-left: 4px solid #FF694B; }
.card-superset { border-left: 4px solid #20A7C9; }
@media (max-width: 768px) {
.header h1 {
font-size: 2rem;
}
.menu-grid {
grid-template-columns: 1fr;
}
}
</style>
</head>
<body>
<div class="container">
<div class="header">
<h1>🏥 Sriphat Data Platform</h1>
<p>Integrated Data Management & Analytics Platform</p>
{% if user %}
<div class="user-info">
<span class="username">👤 {{ user.name or user.username }}</span>
{% if user.roles %}
{% for role in user.roles %}
<span class="role-badge role-{{ role }}">{{ role }}</span>
{% endfor %}
{% endif %}
<a href="{{ root_path }}/auth/logout" class="logout-btn">Logout</a>
</div>
{% endif %}
</div>
<div class="menu-grid">
<!-- Supabase -->
<a href="https://ai.sriphat.com/supabase" class="menu-card card-supabase" target="_blank">
<span class="icon">🗄️</span>
<h3>Supabase</h3>
<p>PostgreSQL database with real-time capabilities and REST API</p>
</a>
<!-- API Docs -->
<a href="{{ root_path }}/docs" class="menu-card card-api">
<span class="icon">📚</span>
<h3>API Documentation</h3>
<p>Interactive API documentation and testing interface</p>
</a>
<!-- Airflow -->
<a href="https://ai.sriphat.com/airflow" class="menu-card card-airflow" target="_blank">
<span class="icon">🔄</span>
<h3>Airflow</h3>
<p>Workflow orchestration and data pipeline management</p>
</a>
<!-- Airbyte -->
<a href="https://ai.sriphat.com/airbyte" class="menu-card card-airbyte" target="_blank">
<span class="icon">🔌</span>
<h3>Airbyte</h3>
<p>Data integration and ETL platform</p>
</a>
<!-- Data Management -->
<div class="menu-card card-data has-submenu" onclick="toggleSubmenu(this)">
<span class="icon">📊</span>
<h3>Data Management <span class="submenu-toggle"></span></h3>
<p>Upload and manage data files</p>
<div class="submenu">
<a href="{{ root_path }}/data-management/finance" class="submenu-item">
💰 Finance Excel Upload
</a>
</div>
</div>
<!-- DBT -->
<a href="https://ai.sriphat.com/dbt" class="menu-card card-dbt" target="_blank">
<span class="icon">🔧</span>
<h3>DBT</h3>
<p>Data transformation and modeling</p>
</a>
<!-- Superset -->
<a href="https://ai.sriphat.com/superset" class="menu-card card-superset" target="_blank">
<span class="icon">📈</span>
<h3>Superset</h3>
<p>Business intelligence and data visualization</p>
</a>
<!-- User Management (Admin only) -->
{% if user and user.roles and 'admin' in user.roles %}
<a href="{{ root_path }}/admin/users" class="menu-card card-admin">
<span class="icon">👥</span>
<h3>User Management</h3>
<p>Manage users and roles (Admin only)</p>
</a>
{% endif %}
<!-- API Management (Admin only) -->
{% if user and user.roles and 'admin' in user.roles %}
<a href="{{ root_path }}/api-management" class="menu-card card-admin">
<span class="icon">🔑</span>
<h3>API Management</h3>
<p>Manage API clients and keys (Admin only)</p>
</a>
{% endif %}
</div>
</div>
<script>
function toggleSubmenu(card) {
const submenu = card.querySelector('.submenu');
const toggle = card.querySelector('.submenu-toggle');
if (submenu.classList.contains('active')) {
submenu.classList.remove('active');
toggle.textContent = '▼';
} else {
submenu.classList.add('active');
toggle.textContent = '▲';
}
}
</script>
</body>
</html>

View File

@@ -48,7 +48,7 @@ async def upsert_to_supabase(
headers["Prefer"] += f",on_conflict={on_conflict}"
try:
async with httpx.AsyncClient(timeout=30.0) as client:
async with httpx.AsyncClient(timeout=30.0, verify=False) as client:
response = await client.post(url, json=data, headers=headers)
response.raise_for_status()
return {
@@ -102,7 +102,7 @@ def upsert_to_supabase_sync(
headers["Prefer"] += f",on_conflict={on_conflict}"
try:
with httpx.Client(timeout=30.0) as client:
with httpx.Client(timeout=30.0, verify=False) as client:
response = client.post(url, json=data, headers=headers)
response.raise_for_status()
return {

View File

@@ -1,6 +1,13 @@
x-common-configs: &common-config
extra_hosts:
- "dev.sriphat.com:192.168.100.9"
- "ai.sriphat.com:192.168.100.8"
pull_policy: ${DOCKER_PULL_POLICY:-missing}
services:
apiservice:
build: .
#build: .
image: 03-apiservice-apiservice:latest
container_name: apiservice
env_file:
- .env
@@ -17,6 +24,18 @@ services:
- ADMIN_SECRET_KEY=${ADMIN_SECRET_KEY}
- ADMIN_USERNAME=${ADMIN_USERNAME}
- ADMIN_PASSWORD=${ADMIN_PASSWORD}
- API_KEY_ENC_SECRET=${ADMIN_SECRET_KEY}
- DEBUG_AUTH=${DEBUG_AUTH:-false}
- KEYCLOAK_SERVER_URL=${KEYCLOAK_SERVER_URL}
- KEYCLOAK_REALM=${KEYCLOAK_REALM}
- KEYCLOAK_CLIENT_ID=${API_KEYCLOAK_CLIENT_ID}
- KEYCLOAK_CLIENT_SECRET=${API_KEYCLOAK_CLIENT_SECRET}
- KEYCLOAK_REDIRECT_URI=${API_KEYCLOAK_REDIRECT_URI}
- MINIO_ENDPOINT=${MINIO_ENDPOINT:-192.168.100.9:9000}
- MINIO_ACCESS_KEY=${MINIO_SVC_ACCESS_KEY}
- MINIO_SECRET_KEY=${MINIO_SVC_SECRET_KEY}
- MINIO_BUCKET_FINANCE=${MINIO_BUCKET_FINANCE:-finance}
- MINIO_USE_SSL=${MINIO_USE_SSL:-false}
- LOG_LEVEL=debug
ports:
- "8040:8040"
@@ -25,6 +44,7 @@ services:
volumes:
- ./app:/app/app
- .env:/app/.env
- ./data/uploads:/data/uploads
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8040/apiservice/docs"]
@@ -32,6 +52,9 @@ services:
timeout: 10s
retries: 3
start_period: 40s
# extra_hosts:
# - "dev.sriphat.com:192.168.100.9"
<<: *common-config
networks:
shared_data_network:

View File

@@ -14,4 +14,7 @@ httpx==0.28.1
WTForms
#==3.2.1
cryptography==42.0.5
python-keycloak==3.9.0
Authlib==1.3.0
python-jose[cryptography]==3.3.0
minio==7.2.11

View File

@@ -40,7 +40,7 @@ might_contain_dag_callable = airflow.utils.file.might_contain_dag_via_default_he
#
# Variable: AIRFLOW__CORE__DEFAULT_TIMEZONE
#
default_timezone = utc
default_timezone = Asia/Bangkok
# The executor class that airflow should use. Choices include
# ``LocalExecutor``, ``CeleryExecutor``,
@@ -90,7 +90,7 @@ simple_auth_manager_all_admins = False
#
# Variable: AIRFLOW__CORE__PARALLELISM
#
parallelism = 8
parallelism = 2
# The maximum number of task instances allowed to run concurrently in each dag run.
# This is also configurable per-dag with ``max_active_tasks``,
@@ -115,7 +115,7 @@ dags_are_paused_at_creation = True
#
# Variable: AIRFLOW__CORE__MAX_ACTIVE_RUNS_PER_DAG
#
max_active_runs_per_dag = 16
max_active_runs_per_dag = 1
# (experimental) The maximum number of consecutive DAG failures before DAG is automatically paused.
# This is also configurable per DAG level with ``max_consecutive_failed_dag_runs``,
@@ -2166,7 +2166,7 @@ refresh_interval = 300
#
# Variable: AIRFLOW__DAG_PROCESSOR__PARSING_PROCESSES
#
parsing_processes = 2
parsing_processes = 1
# One of ``modified_time``, ``random_seeded_by_host`` and ``alphabetical``.
# The DAG processor will list and sort the dag files to decide the parsing order.
@@ -2193,7 +2193,9 @@ max_callbacks_per_loop = 20
#
# Variable: AIRFLOW__DAG_PROCESSOR__MIN_FILE_PROCESS_INTERVAL
#
min_file_process_interval = 30
min_file_process_interval = 90
dag_dir_list_interval = 90
# How long (in seconds) to wait after we have re-parsed a DAG file before deactivating stale
# DAGs (DAGs which are no longer present in the expected files). The reason why we need
@@ -2491,7 +2493,7 @@ flower_basic_auth =
#
# Variable: AIRFLOW__CELERY__SYNC_PARALLELISM
#
sync_parallelism = 0
sync_parallelism = 2
# Import path for celery configuration options
#

View File

@@ -64,7 +64,7 @@ x-airflow-common:
AIRFLOW__CELERY__BROKER_URL: redis://:@redis:6379/0
AIRFLOW__CORE__FERNET_KEY: ${AIRFLOW__CORE__FERNET_KEY:-}
AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION: 'true'
AIRFLOW__CORE__LOAD_EXAMPLES: ${AIRFLOW__CORE__LOAD_EXAMPLES:-'false'}
AIRFLOW__CORE__LOAD_EXAMPLES: ${AIRFLOW__CORE__LOAD_EXAMPLES:-False}
AIRFLOW__CORE__EXECUTION_API_SERVER_URL: 'http://airflow-apiserver:8080/execution/'
# yamllint disable rule:line-length
# Use simple http server on scheduler for health checks
@@ -76,18 +76,22 @@ x-airflow-common:
_PIP_ADDITIONAL_REQUIREMENTS: ${_PIP_ADDITIONAL_REQUIREMENTS:-}
# The following line can be used to set a custom config file, stored in the local config folder
AIRFLOW_CONFIG: '/opt/airflow/config/airflow.cfg'
AIRFLOW__WEBSERVER__BASE_URL: ${AIRFLOW__WEBSERVER__BASE_URL:-https://ai.sriphat.com/airflow}
AIRFLOW__API__BASE_URL: ${AIRFLOW__WEBSERVER__BASE_URL:-https://ai.sriphat.com/airflow}
AIRFLOW__WEBSERVER__WEB_SERVER_PORT: ${AIRFLOW__WEBSERVER__WEB_SERVER_PORT:-8080}
volumes:
- ${AIRFLOW_PROJ_DIR:-.}/dags:/opt/airflow/dags
- ${AIRFLOW_PROJ_DIR:-.}/logs:/opt/airflow/logs
- ${AIRFLOW_PROJ_DIR:-.}/config:/opt/airflow/config
- ${AIRFLOW_PROJ_DIR:-.}/plugins:/opt/airflow/plugins
user: "${AIRFLOW_UID:-50000}:0"
depends_on:
x-depends_on:
&airflow-common-depends-on
#airflow-base:
{}
# airflow-base:
# condition: service_completed_successfully
redis:
condition: service_healthy
# redis:
# condition: service_healthy
networks:
- shared_data_network
@@ -114,19 +118,19 @@ services:
# start_period: 5s
# restart: always
redis:
# Redis is limited to 7.2-bookworm due to licencing change
# https://redis.io/blog/redis-adopts-dual-source-available-licensing/
image: redis:7.2-bookworm
expose:
- 6379
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 30s
retries: 50
start_period: 30s
restart: always
# redis:
# # Redis is limited to 7.2-bookworm due to licencing change
# # https://redis.io/blog/redis-adopts-dual-source-available-licensing/
# image: redis:7.2-bookworm
# expose:
# - 6379
# healthcheck:
# test: ["CMD", "redis-cli", "ping"]
# interval: 10s
# timeout: 30s
# retries: 50
# start_period: 30s
# restart: always
airflow-apiserver:
<<: *airflow-common

View File

@@ -0,0 +1,46 @@
FROM apache/superset:latest
USER root
ENV PATH="/app/.venv/bin:$PATH"
RUN python -m ensurepip --upgrade && /app/.venv/bin/python -m pip install --upgrade pip setuptools wheel
RUN /app/.venv/bin/pip install --no-cache-dir psycopg2-binary
# Install nginx and supervisor
RUN apt-get update && apt-get install -y --no-install-recommends nginx supervisor && rm -rf /var/lib/apt/lists/*
# Copy nginx config
COPY nginx-superset.conf /etc/nginx/sites-available/default
# Create supervisor config to run both nginx and gunicorn
RUN cat > /etc/supervisor/conf.d/supervisord.conf << 'EOF'
[supervisord]
nodaemon=true
user=root
[program:nginx]
command=nginx -g "daemon off;"
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:superset]
command=/app/.venv/bin/gunicorn --bind 127.0.0.1:8088 --workers 4 --timeout 120 --limit-request-line 0 --limit-request-field_size 0 "superset.app:create_app()"
directory=/app
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
user=superset
EOF
EXPOSE 80
USER superset

View File

@@ -0,0 +1,61 @@
server {
listen 80;
server_name localhost;
# Handle API endpoints directly - no rewrite needed
location /superset/api/ {
proxy_pass http://superset:8088/api/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Accept-Encoding "";
proxy_read_timeout 300s;
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
}
# Strip /superset/ prefix, proxy to Superset at root
location /superset/ {
proxy_pass http://superset:8088/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Accept-Encoding "";
proxy_read_timeout 300s;
proxy_connect_timeout 300s;
proxy_send_timeout 300s;
# Rewrite redirect Location headers to add /superset prefix
proxy_redirect ~^(https?://[^/]+)?/(.*)$ $1/superset/$2;
# Rewrite URLs in HTML responses
sub_filter_once off;
sub_filter_types text/html application/json;
sub_filter 'href="/' 'href="/superset/';
sub_filter 'src="/' 'src="/superset/';
sub_filter 'action="/' 'action="/superset/';
sub_filter '"/static/' '"/superset/static/';
sub_filter '"/api/' '"/superset/api/';
sub_filter '"/superset/superset/' '"/superset/superset/';
sub_filter '"/login/' '"/superset/login/';
sub_filter '"/logout/' '"/superset/logout/';
sub_filter '"/lang/' '"/superset/lang/';
}
location = /superset {
return 301 /superset/;
}
# Health check for Docker
location = /health {
proxy_pass http://superset:8088/health;
}
}

View File

@@ -2,9 +2,58 @@ import os
SECRET_KEY = os.environ.get('SUPERSET_SECRET_KEY')
SQLALCHEMY_DATABASE_URI = f"postgresql+psycopg2://{os.environ.get('DATABASE_USER')}:{os.environ.get('DATABASE_PASSWORD')}@{os.environ.get('DATABASE_HOST')}:{os.environ.get('DATABASE_PORT')}/{os.environ.get('DATABASE_DB')}"
ENABLE_PROXY_FIX = True
PUBLIC_ROLE_LIKE = "Gamma"
WTF_CSRF_ENABLED = True
WTF_CSRF_ENABLED = False
WTF_CSRF_TIME_LIMIT = None
FEATURE_FLAGS = {
"EMBEDDED_SUPERSET": True,
}
GUEST_ROLE_NAME = "Gamma"
ENABLE_CORS = True
CORS_OPTIONS = {
'supports_credentials': True,
'allow_headers': ['*'],
'resources': ['*'],
'origins': ['*']
}
SESSION_COOKIE_SAMESITE = "Lax"
SESSION_COOKIE_SECURE = False
GUEST_TOKEN_JWT_SECRET = 'RgSCvATmH8fzluoFB6cqkdCXsY7jjq/zwGLRatoxYtI='
GUEST_TOKEN_JWT_EXP_SECONDS = 86400 # 24 hours
# Logo link configuration
LOGO_TARGET_PATH = '/superset/welcome/'
# Embedded SDK Configuration
EMBEDDED_SUPERSET = True
TALISMAN_ENABLED = False
ENABLE_TEMPLATE_PROCESSING = True
# Guest token configuration for embedded SDK
GUEST_TOKEN_JWT_ALGORITHM = "HS256"
GUEST_TOKEN_JWT_EXP_SECONDS = 300 # 5 minutes
# Domain whitelist for embedded dashboards
WEBDRIVER_BASEURL_USER_FRIENDLY_NAME = "Sriphat Dashboard"
# Embedded SDK Domain Whitelist
EMBEDDED_SDK_HOST_WHITELIST = [
"http://localhost:8800",
"https://ai.sriphat.com",
"http://127.0.0.1:8800"
]
# Allow embedding from specific domains
TALISMAN_ALLOWED_DOMAINS = [
"http://localhost:8800",
"https://ai.sriphat.com",
"http://127.0.0.1:8800"
]

63
07-minio/.env.example Normal file
View File

@@ -0,0 +1,63 @@
# MinIO Configuration
# Copy this file to .env and update with your values
# ============================================================================
# MinIO Credentials
# ============================================================================
MINIO_ROOT_USER=minioadmin
MINIO_ROOT_PASSWORD=minioadmin_secure_password_2026
# ============================================================================
# MinIO Ports
# ============================================================================
MINIO_API_PORT=9000
MINIO_CONSOLE_PORT=9001
# ============================================================================
# MinIO Server URLs (for reverse proxy)
# ============================================================================
# API endpoint URL (for S3 API access)
MINIO_SERVER_URL=https://ai.sriphat.com/minio
# Console UI URL (for web interface)
MINIO_BROWSER_REDIRECT_URL=https://ai.sriphat.com/minio-console
# ============================================================================
# MinIO Region
# ============================================================================
MINIO_REGION=ap-southeast-1
# ============================================================================
# Keycloak Integration (OpenID Connect)
# ============================================================================
# Keycloak OpenID configuration URL
# Format: https://{keycloak-domain}/realms/{realm-name}/.well-known/openid-configuration
MINIO_IDENTITY_OPENID_CONFIG_URL=https://ai.sriphat.com/keycloak/realms/sriphat/.well-known/openid-configuration
# MinIO client in Keycloak
MINIO_IDENTITY_OPENID_CLIENT_ID=minio
# Client secret from Keycloak
MINIO_IDENTITY_OPENID_CLIENT_SECRET=your-minio-client-secret-here
# Claim name for policy mapping (default: policy)
MINIO_IDENTITY_OPENID_CLAIM_NAME=policy
# OpenID scopes
MINIO_IDENTITY_OPENID_SCOPES=openid,profile,email
# Redirect URI after authentication
MINIO_IDENTITY_OPENID_REDIRECT_URI=https://ai.sriphat.com/minio-console/oauth_callback
# ============================================================================
# Service Account — Web Service
# สร้างหลัง MinIO start แล้วด้วย mc CLI
# mc admin user svcacct add --access-key <KEY> --secret-key <SECRET> sriphat admin
# ============================================================================
MINIO_SVC_ACCESS_KEY=sp_service_ac
MINIO_SVC_SECRET_KEY=your-service-account-secret-here
# ============================================================================
# Timezone
# ============================================================================
TZ=Asia/Bangkok

30
07-minio/.gitignore vendored Normal file
View File

@@ -0,0 +1,30 @@
# Environment variables
.env
# Data directory (persistent storage)
data/
# SSL certificates
certs/
# Logs
*.log
# Backup files
*.tar.gz
*.zip
# Temporary files
*.tmp
*.temp
# OS files
.DS_Store
Thumbs.db
# IDE files
.vscode/
.idea/
*.swp
*.swo
*~

View File

@@ -0,0 +1,362 @@
# MinIO Keycloak Integration Guide
Complete guide for integrating MinIO with Keycloak for SSO authentication.
## 🎯 Overview
MinIO supports OpenID Connect (OIDC) authentication, allowing users to log in to MinIO Console using Keycloak credentials. This integration provides:
- **Single Sign-On (SSO)** - Users authenticate once with Keycloak
- **Centralized User Management** - Manage users in Keycloak
- **Role-Based Access Control** - Map Keycloak roles to MinIO policies
- **Secure Authentication** - OAuth 2.0 / OpenID Connect flow
## 📋 Prerequisites
- Keycloak instance running and accessible
- MinIO instance running
- Admin access to both Keycloak and MinIO
## 🔧 Setup Steps
### **Step 1: Create MinIO Client in Keycloak**
1. **Login to Keycloak Admin Console**
```
https://ai.sriphat.com/keycloak
```
2. **Select Realm**
- Go to your realm (e.g., `sriphat`)
3. **Create Client**
- Navigate to: **Clients** → **Create Client**
- **Client ID**: `minio`
- **Client Type**: `OpenID Connect`
- **Client Protocol**: `openid-connect`
- Click **Next**
4. **Capability Config**
- **Client authentication**: `ON`
- **Authorization**: `OFF`
- **Authentication flow**:
- ✅ Standard flow
- ✅ Direct access grants
- ❌ Implicit flow
- ❌ Service accounts roles
- Click **Next**
5. **Login Settings**
- **Root URL**: `https://ai.sriphat.com/minio-console`
- **Home URL**: `https://ai.sriphat.com/minio-console`
- **Valid redirect URIs**:
```
https://ai.sriphat.com/minio-console/*
https://ai.sriphat.com/minio-console/oauth_callback
```
- **Valid post logout redirect URIs**: `https://ai.sriphat.com/minio-console`
- **Web origins**: `https://ai.sriphat.com`
- Click **Save**
6. **Get Client Secret**
- Go to **Credentials** tab
- Copy the **Client Secret**
- Save this for `.env` configuration
### **Step 2: Create Client Scope for MinIO Policy**
1. **Create Client Scope**
- Navigate to: **Client Scopes** → **Create client scope**
- **Name**: `minio-authorization`
- **Type**: `Optional`
- **Protocol**: `OpenID Connect`
- **Display on consent screen**: `OFF`
- Click **Save**
2. **Add Mapper for Policy Claim**
- Go to **Mappers** tab
- Click **Add mapper** → **By configuration**
- Select **User Attribute**
- **Name**: `minio-policy`
- **User Attribute**: `minio_policy`
- **Token Claim Name**: `policy`
- **Claim JSON Type**: `String`
- **Add to ID token**: `ON`
- **Add to access token**: `ON`
- **Add to userinfo**: `ON`
- Click **Save**
3. **Assign Scope to MinIO Client**
- Go to **Clients** → `minio`
- Go to **Client scopes** tab
- Click **Add client scope**
- Select `minio-authorization`
- Choose **Optional**
- Click **Add**
### **Step 3: Create MinIO Policies in Keycloak**
MinIO uses policies to control access. Common policies:
- `consoleAdmin` - Full admin access
- `readonly` - Read-only access
- `readwrite` - Read and write access
- `diagnostics` - Diagnostics access
**Add Policy to Users:**
1. **Go to Users**
- Navigate to: **Users** → Select user
2. **Add Attribute**
- Go to **Attributes** tab
- Click **Add attribute**
- **Key**: `minio_policy`
- **Value**: `consoleAdmin` (or other policy)
- Click **Save**
### **Step 4: Configure MinIO Environment Variables**
Update `07-minio/.env`:
```bash
# Keycloak Integration
MINIO_IDENTITY_OPENID_CONFIG_URL=https://ai.sriphat.com/keycloak/realms/sriphat/.well-known/openid-configuration
MINIO_IDENTITY_OPENID_CLIENT_ID=minio
MINIO_IDENTITY_OPENID_CLIENT_SECRET=your-client-secret-from-step-1
MINIO_IDENTITY_OPENID_CLAIM_NAME=policy
MINIO_IDENTITY_OPENID_SCOPES=openid,profile,email,minio-authorization
MINIO_IDENTITY_OPENID_REDIRECT_URI=https://ai.sriphat.com/minio-console/oauth_callback
```
### **Step 5: Restart MinIO**
```bash
cd 07-minio
docker compose down
docker compose up -d
```
### **Step 6: Test Authentication**
1. **Access MinIO Console**
```
https://ai.sriphat.com/minio-console
```
2. **Click "Login with SSO"**
- You'll be redirected to Keycloak
- Login with Keycloak credentials
- After successful authentication, you'll be redirected back to MinIO Console
## 🔐 MinIO Policies
### **Default Policies**
MinIO comes with built-in policies:
| Policy | Description |
|--------|-------------|
| `consoleAdmin` | Full admin access to console and buckets |
| `readonly` | Read-only access to buckets |
| `readwrite` | Read and write access to buckets |
| `diagnostics` | Access to diagnostics and monitoring |
| `writeonly` | Write-only access (upload only) |
### **Custom Policies**
Create custom policies in MinIO Console or via `mc` CLI:
```bash
# Install mc (MinIO Client)
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
sudo mv mc /usr/local/bin/
# Configure mc
mc alias set myminio https://ai.sriphat.com/minio minioadmin minioadmin_secure_password_2026
# Create custom policy
cat > custom-policy.json <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::mybucket/*"
]
}
]
}
EOF
# Add policy to MinIO
mc admin policy create myminio custom-policy custom-policy.json
```
## 🔄 Policy Mapping Flow
```
User logs in with Keycloak
Keycloak returns ID token with 'policy' claim
MinIO reads 'policy' claim value (e.g., "consoleAdmin")
MinIO applies corresponding policy to user session
User has permissions defined by the policy
```
## 🎯 Role-Based Access Example
### **Scenario: Different User Roles**
**Admin Users:**
```
Keycloak User Attribute:
minio_policy: consoleAdmin
```
**Data Scientists:**
```
Keycloak User Attribute:
minio_policy: readwrite
```
**Analysts:**
```
Keycloak User Attribute:
minio_policy: readonly
```
## 🐛 Troubleshooting
### **Issue: "Login with SSO" button not showing**
**Check:**
```bash
# Verify environment variables
docker exec minio printenv | grep MINIO_IDENTITY_OPENID
# Check MinIO logs
docker logs minio
```
**Solution:**
- Ensure all `MINIO_IDENTITY_OPENID_*` variables are set
- Restart MinIO container
### **Issue: Redirect loop after login**
**Check:**
- `MINIO_BROWSER_REDIRECT_URL` matches Keycloak redirect URI
- Valid redirect URIs in Keycloak client include `/oauth_callback`
**Solution:**
```bash
# Update .env
MINIO_BROWSER_REDIRECT_URL=https://ai.sriphat.com/minio-console
MINIO_IDENTITY_OPENID_REDIRECT_URI=https://ai.sriphat.com/minio-console/oauth_callback
```
### **Issue: User has no permissions after login**
**Check:**
- User has `minio_policy` attribute in Keycloak
- Policy name matches MinIO policy exactly (case-sensitive)
**Solution:**
```bash
# Verify user attribute in Keycloak
# Add minio_policy attribute with value: consoleAdmin
```
### **Issue: Cannot access Keycloak config URL**
**Check:**
```bash
# Test from MinIO container
docker exec minio curl -k https://ai.sriphat.com/keycloak/realms/sriphat/.well-known/openid-configuration
```
**Solution:**
- Ensure MinIO container can reach Keycloak
- Check network connectivity
- Verify Keycloak realm name is correct
## 📊 Monitoring
### **Check OpenID Configuration**
```bash
# View current OpenID config
docker exec minio mc admin config get myminio identity_openid
```
### **View Active Sessions**
```bash
# List active user sessions
docker exec minio mc admin user list myminio
```
### **Audit Logs**
```bash
# Enable audit logging
docker exec minio mc admin config set myminio audit_webhook:1 endpoint="http://your-webhook-endpoint"
# View logs
docker logs minio -f
```
## 🔒 Security Best Practices
1. **Use HTTPS Only**
- Always use HTTPS for MinIO and Keycloak
- Configure SSL certificates properly
2. **Rotate Client Secrets**
- Periodically rotate Keycloak client secrets
- Update MinIO configuration after rotation
3. **Least Privilege Principle**
- Assign minimal required policies to users
- Use custom policies for specific use cases
4. **Monitor Access**
- Enable audit logging
- Review access logs regularly
5. **Secure Network**
- Use firewall rules to restrict access
- Consider VPN for sensitive data
## 📚 References
- [MinIO OpenID Connect](https://min.io/docs/minio/linux/operations/external-iam/configure-openid-external-identity-management.html)
- [Keycloak OpenID Connect](https://www.keycloak.org/docs/latest/server_admin/#_oidc)
- [MinIO IAM Policies](https://min.io/docs/minio/linux/administration/identity-access-management/policy-based-access-control.html)
## 🎉 Summary
After completing these steps:
- ✅ MinIO integrated with Keycloak SSO
- ✅ Users can login with Keycloak credentials
- ✅ Role-based access control configured
- ✅ Centralized user management in Keycloak
- ✅ Secure HTTPS access via Nginx reverse proxy
**Access MinIO Console:**
```
https://ai.sriphat.com/minio-console
```
**Login with SSO** → Keycloak authentication → MinIO Console access! 🚀

647
07-minio/README.md Normal file
View File

@@ -0,0 +1,647 @@
# MinIO Object Storage Service
MinIO is a high-performance, S3-compatible object storage system. This setup includes persistent storage, HTTPS access via Nginx reverse proxy, and Keycloak SSO integration.
## 🎯 Overview
**MinIO Features:**
- **S3-Compatible API** - Works with AWS S3 SDKs and tools
- **High Performance** - Optimized for large-scale data workloads
- **Distributed Storage** - Supports multi-node deployment
- **Web Console** - User-friendly web interface
- **Encryption** - Server-side and client-side encryption
- **Versioning** - Object versioning support
- **Lifecycle Management** - Automatic data retention policies
**This Setup Includes:**
- Docker Compose configuration
- Persistent storage with volume mounts
- HTTPS access via Nginx reverse proxy
- Keycloak SSO integration (OpenID Connect)
- Health checks and monitoring
## 📋 Prerequisites
- Docker and Docker Compose installed
- Network: `shared_data_network` created
- Nginx reverse proxy configured
- Keycloak instance running (for SSO)
- Server: 192.168.100.9
## 🚀 Quick Start
### **Step 1: Configure Environment**
```bash
cd 07-minio
# Copy example environment file
cp .env.example .env
# Edit .env with your settings
nano .env
```
**Required Configuration:**
```bash
# MinIO Credentials
MINIO_ROOT_USER=minioadmin
MINIO_ROOT_PASSWORD=your-secure-password-here
# Keycloak Integration
MINIO_IDENTITY_OPENID_CLIENT_SECRET=your-keycloak-client-secret
```
### **Step 2: Create Data Directory**
```bash
# Create persistent storage directory
mkdir -p data
# Set permissions
chmod 755 data
```
### **Step 3: Start MinIO**
```bash
# Start service
docker compose up -d
# Check status
docker compose ps
# View logs
docker logs minio -f
```
### **Step 4: Configure Nginx Reverse Proxy**
Add the configuration from `nginx-minio.conf` to your Nginx Proxy Manager:
1. Go to Nginx Proxy Manager UI
2. Create/Edit Proxy Host for `ai.sriphat.com`
3. Add MinIO configuration to "Custom Nginx Configuration"
4. Save and test
### **Step 5: Setup Keycloak Integration**
Follow the detailed guide in `KEYCLOAK_INTEGRATION.md`:
1. Create MinIO client in Keycloak
2. Configure client scopes and mappers
3. Add policy attributes to users
4. Update MinIO environment variables
5. Restart MinIO service
## 🌐 Access URLs
**MinIO Console (Web UI):**
```
https://ai.sriphat.com/minio-console
```
**MinIO API (S3 Compatible):**
```
https://ai.sriphat.com/minio
```
**Direct Access (Development):**
```
http://192.168.100.9:9001 (Console)
http://192.168.100.9:9000 (API)
```
## 🔑 Authentication
### **Option 1: Root Credentials (Default)**
Login with root credentials from `.env`:
- **Username**: Value of `MINIO_ROOT_USER`
- **Password**: Value of `MINIO_ROOT_PASSWORD`
### **Option 2: Keycloak SSO (Recommended)**
1. Click "Login with SSO" on MinIO Console
2. Authenticate with Keycloak
3. Access granted based on policy mapping
See `KEYCLOAK_INTEGRATION.md` for setup instructions.
## 📦 Using MinIO
### **Web Console**
1. Access: `https://ai.sriphat.com/minio-console`
2. Login with credentials or SSO
3. Create buckets, upload files, manage access
### **MinIO Client (mc)**
```bash
# Install mc
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
sudo mv mc /usr/local/bin/
# Configure alias
mc alias set myminio https://ai.sriphat.com/minio minioadmin your-password
# List buckets
mc ls myminio
# Create bucket
mc mb myminio/my-bucket
# Upload file
mc cp myfile.txt myminio/my-bucket/
# Download file
mc cp myminio/my-bucket/myfile.txt ./
# List objects
mc ls myminio/my-bucket
# Remove object
mc rm myminio/my-bucket/myfile.txt
```
### **Python SDK (minio — แนะนำสำหรับ Sriphat Platform)**
ใช้ `minio` package (Official MinIO Python SDK) แทน boto3 สำหรับ internal services:
```python
from minio import Minio
import io
# Connection — ใช้ internal IP จาก service บน server อื่น
client = Minio(
endpoint="192.168.100.9:9000", # internal IP, ไม่ใช่ public URL
access_key="sp_service_ac",
secret_key="<MINIO_SVC_SECRET_KEY>",
secure=False, # HTTP ภายใน network
)
# Upload file
with open("report.xlsx", "rb") as f:
data = f.read()
client.put_object(
bucket_name="finance",
object_name="finance/20260520_report.xlsx",
data=io.BytesIO(data),
length=len(data),
content_type="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
)
# Download file (คืนค่าเป็น HTTPResponse)
response = client.get_object("finance", "finance/20260520_report.xlsx")
data = response.read()
response.close()
response.release_conn()
# Download ไปยัง file โดยตรง
client.fget_object("finance", "finance/20260520_report.xlsx", "/tmp/report.xlsx")
# List objects ใน bucket
for obj in client.list_objects("finance", prefix="finance/", recursive=True):
print(obj.object_name, obj.size)
# Generate presigned URL (สำหรับให้ภายนอกดาวน์โหลด ใช้ได้ 1 ชั่วโมง)
from datetime import timedelta
url = client.presigned_get_object("finance", "finance/report.xlsx", expires=timedelta(hours=1))
print(url)
```
---
### **Airflow DAG — อ่านไฟล์จาก MinIO finance bucket**
Airflow อยู่บน server .9 (server เดียวกับ MinIO) ใช้ container name `minio:9000` หรือ `192.168.100.9:9000`:
```python
from minio import Minio
import pandas as pd
import io
from airflow.decorators import dag, task
from airflow.utils.dates import days_ago
@dag(schedule=None, start_date=days_ago(1), catchup=False)
def process_finance_excel():
@task
def download_and_process(filepath: str, **context):
"""
filepath = MinIO object key เช่น "finance/20260520_123000_report.xlsx"
ส่งมาจาก API Service ผ่าน DAG trigger conf
"""
client = Minio(
endpoint="minio:9000", # container name บน shared_data_network
access_key="sp_service_ac", # ใช้ service account เดียวกับ API
secret_key="{{ var.value.MINIO_SVC_SECRET_KEY }}", # เก็บใน Airflow Variables
secure=False,
)
# Download file จาก MinIO
response = client.get_object(bucket_name="finance", object_name=filepath)
file_bytes = response.read()
response.close()
response.release_conn()
# ประมวลผลด้วย pandas
df = pd.read_excel(io.BytesIO(file_bytes))
print(f"Loaded {len(df)} rows from {filepath}")
# ... process data ...
return {"rows": len(df), "filepath": filepath}
@task
def get_filepath(**context):
conf = context["dag_run"].conf or {}
return conf.get("filepath", "")
fp = get_filepath()
download_and_process(fp)
process_finance_excel()
```
**ตั้งค่า Airflow Connection (ทางเลือก — ใช้ S3Hook)**
ถ้าต้องการใช้ `S3Hook` หรือ Airflow Operators:
```
Connection ID : minio_s3
Connection Type: Amazon Web Services
Extra (JSON) : {"endpoint_url": "http://minio:9000", "region_name": "ap-southeast-1"}
Login : sp_service_ac
Password : <MINIO_SVC_SECRET_KEY>
```
```python
from airflow.providers.amazon.aws.hooks.s3 import S3Hook
hook = S3Hook(aws_conn_id="minio_s3")
obj = hook.get_key(key="finance/20260520_report.xlsx", bucket_name="finance")
data = obj.get()["Body"].read()
df = pd.read_excel(io.BytesIO(data))
```
**Airflow Variables ที่ต้องสร้าง:**
| Key | Value |
|-----|-------|
| `MINIO_ENDPOINT` | `192.168.100.9:9000` |
| `MINIO_SVC_SECRET_KEY` | (ดูจาก `07-minio/.env`) |
---
### **Python SDK (boto3)**
```python
import boto3
from botocore.client import Config
# Configure S3 client
s3 = boto3.client(
's3',
endpoint_url='https://ai.sriphat.com/minio',
aws_access_key_id='minioadmin',
aws_secret_access_key='your-password',
config=Config(signature_version='s3v4'),
region_name='ap-southeast-1'
)
# List buckets
response = s3.list_buckets()
for bucket in response['Buckets']:
print(bucket['Name'])
# Upload file
s3.upload_file('myfile.txt', 'my-bucket', 'myfile.txt')
# Download file
s3.download_file('my-bucket', 'myfile.txt', 'downloaded.txt')
# List objects
response = s3.list_objects_v2(Bucket='my-bucket')
for obj in response.get('Contents', []):
print(obj['Key'])
```
### **AWS CLI**
```bash
# Configure AWS CLI
aws configure set aws_access_key_id minioadmin
aws configure set aws_secret_access_key your-password
aws configure set region ap-southeast-1
# List buckets
aws --endpoint-url https://ai.sriphat.com/minio s3 ls
# Create bucket
aws --endpoint-url https://ai.sriphat.com/minio s3 mb s3://my-bucket
# Upload file
aws --endpoint-url https://ai.sriphat.com/minio s3 cp myfile.txt s3://my-bucket/
# Download file
aws --endpoint-url https://ai.sriphat.com/minio s3 cp s3://my-bucket/myfile.txt ./
# Sync directory
aws --endpoint-url https://ai.sriphat.com/minio s3 sync ./mydir s3://my-bucket/mydir/
```
## 🔧 Configuration
### **Environment Variables**
| Variable | Description | Default |
|----------|-------------|---------|
| `MINIO_ROOT_USER` | Root username | minioadmin |
| `MINIO_ROOT_PASSWORD` | Root password | - |
| `MINIO_API_PORT` | API port | 9000 |
| `MINIO_CONSOLE_PORT` | Console port | 9001 |
| `MINIO_SERVER_URL` | API endpoint URL | - |
| `MINIO_BROWSER_REDIRECT_URL` | Console URL | - |
| `MINIO_REGION` | Default region | ap-southeast-1 |
### **Keycloak Integration**
| Variable | Description |
|----------|-------------|
| `MINIO_IDENTITY_OPENID_CONFIG_URL` | Keycloak OIDC config URL |
| `MINIO_IDENTITY_OPENID_CLIENT_ID` | Client ID in Keycloak |
| `MINIO_IDENTITY_OPENID_CLIENT_SECRET` | Client secret |
| `MINIO_IDENTITY_OPENID_CLAIM_NAME` | Policy claim name |
| `MINIO_IDENTITY_OPENID_SCOPES` | OIDC scopes |
### **Storage**
**Persistent Data:**
```
07-minio/data/ # Object storage data
07-minio/certs/ # SSL certificates (optional)
```
**Volume Mounts:**
```yaml
volumes:
- ./data:/data # Storage data
- ./certs:/root/.minio/certs:ro # SSL certs
```
## 🔒 Security
### **1. Strong Passwords**
```bash
# Generate strong password
openssl rand -base64 32
# Update .env
MINIO_ROOT_PASSWORD=generated-password-here
```
### **2. Network Security**
```bash
# Firewall rules (if needed)
sudo ufw allow from 192.168.100.0/24 to any port 9000
sudo ufw allow from 192.168.100.0/24 to any port 9001
```
### **3. HTTPS Only**
- Always use HTTPS in production
- Configure SSL certificates in Nginx
- Set `MINIO_SERVER_URL` and `MINIO_BROWSER_REDIRECT_URL` to HTTPS
### **4. Access Policies**
```bash
# Create read-only policy
mc admin policy create myminio readonly-policy readonly-policy.json
# Assign policy to user
mc admin policy attach myminio readonly-policy --user=username
```
### **5. Bucket Policies**
```bash
# Set bucket policy (public read)
mc anonymous set download myminio/public-bucket
# Set bucket policy (private)
mc anonymous set none myminio/private-bucket
```
## 📊 Monitoring
### **Health Check**
```bash
# Check MinIO health
curl -k https://ai.sriphat.com/minio/health/live
# Check from container
docker exec minio curl -f http://localhost:9000/minio/health/live
```
### **Logs**
```bash
# View logs
docker logs minio -f
# View last 100 lines
docker logs minio --tail 100
# Export logs
docker logs minio > minio.log
```
### **Metrics**
```bash
# View server info
mc admin info myminio
# View server stats
mc admin prometheus metrics myminio
```
### **Disk Usage**
```bash
# Check disk usage
mc admin info myminio
# Check bucket size
mc du myminio/my-bucket
```
## 🐛 Troubleshooting
### **Issue: Cannot access MinIO Console**
**Check:**
```bash
# Verify container is running
docker ps | grep minio
# Check logs
docker logs minio
# Test direct access
curl http://192.168.100.9:9001
```
**Solution:**
- Ensure container is running: `docker compose up -d`
- Check firewall rules
- Verify Nginx configuration
### **Issue: SSO login not working**
**Check:**
```bash
# Verify Keycloak config
docker exec minio printenv | grep MINIO_IDENTITY_OPENID
# Test Keycloak connectivity
docker exec minio curl -k https://ai.sriphat.com/keycloak/realms/sriphat/.well-known/openid-configuration
```
**Solution:**
- Verify all Keycloak environment variables are set
- Check client secret is correct
- Ensure redirect URIs match in Keycloak
- See `KEYCLOAK_INTEGRATION.md` for detailed troubleshooting
### **Issue: Upload fails**
**Check:**
```bash
# Check disk space
df -h
# Check permissions
ls -la data/
```
**Solution:**
- Ensure sufficient disk space
- Check directory permissions: `chmod 755 data/`
- Increase `client_max_body_size` in Nginx
### **Issue: S3 API connection refused**
**Check:**
```bash
# Test API endpoint
curl -k https://ai.sriphat.com/minio/
# Test direct connection
curl http://192.168.100.9:9000/
```
**Solution:**
- Verify `MINIO_SERVER_URL` is set correctly
- Check Nginx proxy configuration
- Ensure port 9000 is accessible
## 🔄 Maintenance
### **Backup**
```bash
# Backup data directory
tar -czf minio-backup-$(date +%Y%m%d).tar.gz data/
# Backup to remote location
rsync -avz data/ user@backup-server:/backups/minio/
```
### **Update MinIO**
```bash
# Pull latest image
docker compose pull
# Restart with new image
docker compose up -d
# Verify version
docker exec minio minio --version
```
### **Restore**
```bash
# Stop MinIO
docker compose down
# Restore data
tar -xzf minio-backup-20260325.tar.gz
# Start MinIO
docker compose up -d
```
## 📚 Documentation
- **MinIO Official Docs**: https://min.io/docs/minio/linux/
- **S3 API Reference**: https://docs.aws.amazon.com/AmazonS3/latest/API/
- **Keycloak Integration**: See `KEYCLOAK_INTEGRATION.md`
- **Nginx Configuration**: See `nginx-minio.conf`
## 🎯 Use Cases
### **1. Data Lake Storage**
- Store raw data files (CSV, JSON, Parquet)
- Integrate with Spark, Pandas, Dask
- Version control for datasets
### **2. Backup Storage**
- Database backups
- Application backups
- Log archival
### **3. Media Storage**
- Images, videos, documents
- CDN integration
- Static website hosting
### **4. ML/AI Workflows**
- Model storage
- Training data storage
- Experiment artifacts
### **5. Application Storage**
- User uploads
- Generated reports
- Temporary files
## 🎉 Summary
**What You Have:**
- ✅ MinIO object storage service
- ✅ Persistent storage with volume mounts
- ✅ HTTPS access via Nginx reverse proxy
- ✅ Keycloak SSO integration ready
- ✅ S3-compatible API
- ✅ Web console for management
- ✅ Health checks and monitoring
**Access:**
- Console: `https://ai.sriphat.com/minio-console`
- API: `https://ai.sriphat.com/minio`
**Next Steps:**
1. Configure `.env` file
2. Start MinIO: `docker compose up -d`
3. Setup Keycloak integration (optional)
4. Configure Nginx reverse proxy
5. Create buckets and start using!
For detailed Keycloak SSO setup, see `KEYCLOAK_INTEGRATION.md` 🚀

View File

@@ -0,0 +1,50 @@
version: '3.8'
services:
minio:
image: minio/minio:latest
container_name: minio
command: server /data --console-address ":9001"
ports:
- "${MINIO_API_PORT:-9000}:9000"
- "${MINIO_CONSOLE_PORT:-9001}:9001"
environment:
# MinIO credentials
MINIO_ROOT_USER: ${MINIO_ROOT_USER}
MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}
# Server settings
MINIO_SERVER_URL: ${MINIO_SERVER_URL:-https://ai.sriphat.com/minio}
MINIO_BROWSER_REDIRECT_URL: ${MINIO_BROWSER_REDIRECT_URL:-https://ai.sriphat.com/minio-console}
# Region
MINIO_REGION: ${MINIO_REGION:-ap-southeast-1}
# Identity OpenID (Keycloak)
MINIO_IDENTITY_OPENID_CONFIG_URL: ${MINIO_IDENTITY_OPENID_CONFIG_URL}
MINIO_IDENTITY_OPENID_CLIENT_ID: ${MINIO_IDENTITY_OPENID_CLIENT_ID}
MINIO_IDENTITY_OPENID_CLIENT_SECRET: ${MINIO_IDENTITY_OPENID_CLIENT_SECRET}
MINIO_IDENTITY_OPENID_CLAIM_NAME: ${MINIO_IDENTITY_OPENID_CLAIM_NAME:-policy}
MINIO_IDENTITY_OPENID_SCOPES: ${MINIO_IDENTITY_OPENID_SCOPES:-openid,profile,email}
MINIO_IDENTITY_OPENID_REDIRECT_URI: ${MINIO_IDENTITY_OPENID_REDIRECT_URI}
# Timezone
TZ: ${TZ:-Asia/Bangkok}
volumes:
# Persistent storage
- ./data:/data
# SSL certificates (if using direct HTTPS)
- ./certs:/root/.minio/certs:ro
networks:
- shared_data_network
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 10s
retries: 3
start_period: 10s
networks:
shared_data_network:
external: true

104
07-minio/nginx-minio.conf Normal file
View File

@@ -0,0 +1,104 @@
# MinIO Nginx Configuration
# For use with Nginx Proxy Manager or standalone Nginx
# This configuration provides HTTPS access to MinIO API and Console
# ============================================================================
# MinIO S3 API - Port 9000
# Subpath: /minio
# ============================================================================
location /minio/ {
# Rewrite path to remove /minio prefix
rewrite ^/minio/(.*) /$1 break;
# Forward to MinIO API
proxy_pass http://192.168.100.9:9000;
# Preserve headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
# Required for MinIO
proxy_set_header X-NginX-Proxy true;
# Disable buffering for large uploads
proxy_buffering off;
proxy_request_buffering off;
# Timeouts for large file uploads/downloads
proxy_connect_timeout 300;
proxy_send_timeout 300;
proxy_read_timeout 300;
send_timeout 300;
# Max upload size (adjust as needed)
client_max_body_size 0;
}
# ============================================================================
# MinIO Console (Web UI) - Port 9001
# Subpath: /minio-console
# ============================================================================
location /minio-console/ {
# Forward to MinIO Console
proxy_pass http://192.168.100.9:9001/;
# Preserve headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
# WebSocket support for real-time updates
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Disable buffering
proxy_buffering off;
proxy_request_buffering off;
# Timeouts
proxy_connect_timeout 300;
proxy_send_timeout 300;
proxy_read_timeout 300;
}
# ============================================================================
# MinIO Console Assets
# ============================================================================
location /minio-console/assets/ {
proxy_pass http://192.168.100.9:9001/assets/;
proxy_set_header Host $host;
proxy_cache_valid 200 1d;
add_header Cache-Control "public, immutable";
}
# ============================================================================
# MinIO Console API
# ============================================================================
location /minio-console/api/ {
proxy_pass http://192.168.100.9:9001/api/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# WebSocket support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_buffering off;
}
# ============================================================================
# MinIO Health Check
# ============================================================================
location /minio/health {
proxy_pass http://192.168.100.9:9000/minio/health;
proxy_set_header Host $host;
access_log off;
}

55
CLAUDE.md Normal file
View File

@@ -0,0 +1,55 @@
# Sriphat Data Platform — Claude Context
## Project Paths
| Environment | Path |
|-------------|------|
| **Local (WSL2)** | `/mnt/e/git3/sriphat-dataplatform` |
| **Server .8** | `/home/bdadmin/sriphat-dataplatform` |
| **Server .9** | `/home/bdadmin/sriphat-dataplatform` (เฉพาะ folders ของ service ที่รันบนเครื่องนี้) |
| **Windows** | `E:\git3\sriphat-dataplatform` |
## Remote Hosts (WSL2 SSH)
| Server | IP | SSH Script |
|--------|----|-----------|
| **Server 1** | 192.168.100.8 | `~/key/ssh_sriphat_8.sh` |
| **Server 2** | 192.168.100.9 | `~/key/ssh_sriphat_9.sh` |
## Service Distribution
### Server 1 — 192.168.100.8 (folders ทั้งหมด)
```
00-network/ # Docker network setup
01-infra/ # Nginx, Keycloak, PostgreSQL, Redis, Dozzle
02-supabase/ # Supabase full stack (13 containers)
03-apiservice/ # Custom FastAPI service
06-analytics/ # Apache Superset
```
### Server 2 — 192.168.100.9 (เฉพาะ folders ของ service ที่ใช้บนเครื่องนี้)
```
05-airflow/ # Apache Airflow (CeleryExecutor)
07-minio/ # MinIO Object Storage
```
> **หมายเหตุ:** OpenMetadata อยู่บน server .9 แต่ไม่อยู่ใน repo นี้
## Global Environment File
- **Local dev:** `.env.global` (root of project)
- **Server:** `/home/bdadmin/sriphat-dataplatform/.env.global`
- ไฟล์นี้อยู่ใน `.gitignore` — ต้องสร้างและ sync เองบนแต่ละเครื่อง
## Docker Network
ทุก service ใช้ network ร่วม ต้องสร้างก่อนรัน:
```bash
docker network create shared_data_network
```
## Daily Log Convention
- Path: `_daily-log/YYYY-MM-DD-sriphat-dataplatform.md`
- Folder `_daily-log/` อยู่ใน `.gitignore`
- ดู convention ที่: `_daily-log/` (ถ้ามี README) หรือจาก BDA standard

View File

@@ -0,0 +1,400 @@
# Dozzle Multi-Host Setup Guide
คู่มือการตั้งค่า Dozzle สำหรับ monitor Docker containers บนหลาย hosts
## 🏗️ Architecture Overview
```
┌─────────────────────────────────────────────────────────────┐
│ Main Server (Current Host) │
│ ├─ Nginx Proxy Manager │
│ ├─ Keycloak │
│ ├─ PostgreSQL │
│ ├─ API Service │
│ ├─ Supabase │
│ ├─ Superset │
│ └─ Dozzle (Main UI) ──────────────┐ │
└───────────────────────────────────┼─────────────────────────┘
┌───────────────┴───────────────┐
│ │
┌───────────▼──────────┐ ┌────────────▼─────────┐
│ 192.168.100.9 │ │ 192.168.100.9 │
│ Airbyte Host │ │ Airflow Host │
│ ├─ Airbyte Services │ │ ├─ Airflow Services │
│ └─ Dozzle Agent │ │ └─ Dozzle Agent │
│ (Port 7007) │ │ (Port 7008) │
└──────────────────────┘ └──────────────────────┘
```
## 📋 Setup Steps
### **Step 1: ติดตั้ง Dozzle Agent บน Remote Hosts**
#### **สำหรับ Airbyte Host (192.168.100.9:7007)**
สร้าง/แก้ไข `docker-compose.yml` ใน Airbyte directory:
```yaml
services:
# ... existing Airbyte services ...
dozzle-agent:
image: amir20/dozzle:latest
container_name: dozzle-agent-airbyte
command: agent
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
ports:
- "7007:7007"
environment:
DOZZLE_LEVEL: info
DOZZLE_HOSTNAME: Airbyte Server
TZ: Asia/Bangkok
restart: unless-stopped
networks:
- airbyte_network # ใช้ network ของ Airbyte
```
**Start agent:**
```bash
docker compose up -d dozzle-agent
```
#### **สำหรับ Airflow Host (192.168.100.9:7008)**
สร้าง/แก้ไข `docker-compose.yml` ใน Airflow directory:
```yaml
services:
# ... existing Airflow services ...
dozzle-agent:
image: amir20/dozzle:latest
container_name: dozzle-agent-airflow
command: agent
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
ports:
- "7008:7007" # External: 7008, Internal: 7007
environment:
DOZZLE_LEVEL: info
DOZZLE_HOSTNAME: Airflow Server
TZ: Asia/Bangkok
restart: unless-stopped
networks:
- shared_data_network # ใช้ network ของ Airflow
```
**Start agent:**
```bash
docker compose up -d dozzle-agent
```
### **Step 2: ตรวจสอบ Agents**
```bash
# ตรวจสอบ Airbyte agent
curl http://192.168.100.9:7007/healthcheck
# ตรวจสอบ Airflow agent
curl http://192.168.100.9:7008/healthcheck
# ดู logs
docker logs dozzle-agent-airbyte
docker logs dozzle-agent-airflow
```
### **Step 3: Start Dozzle Main UI (Main Server)**
```bash
cd 01-infra
docker compose up -d dozzle
# ตรวจสอบ
docker logs dozzle -f
```
### **Step 4: เข้าใช้งาน**
**Direct access:**
```
http://localhost:9999/dozzle
```
**ผ่าน Nginx:**
```
http://ai.sriphat.com/dozzle
```
## 🔧 Configuration Details
### **Main Server (.env.global)**
```bash
# Dozzle - Docker Log Viewer & Monitoring
DOZZLE_PORT=9999
DOZZLE_LEVEL=info
DOZZLE_BASE=/dozzle
DOZZLE_HOSTNAME=Sriphat Main Server
DOZZLE_AUTH_PROVIDER=none
DOZZLE_RESTART_POLICY=unless-stopped
# Remote agents: Airbyte and Airflow on 192.168.100.9
# Format: host:port,host:port (comma-separated)
DOZZLE_REMOTE_AGENT=192.168.100.9:7007,192.168.100.9:7008
```
### **Agent Configuration**
**Airbyte Agent:**
- Port: 7007
- Hostname: Airbyte Server
- Monitors: Airbyte containers
**Airflow Agent:**
- Port: 7008
- Hostname: Airflow Server
- Monitors: Airflow containers
## 🌐 Nginx Configuration
Dozzle config ถูกเพิ่มใน:
- `01-infra/nginx-configs/dozzle.conf`
- `01-infra/nginx-configs/complete-example.conf`
**ตั้งค่าใน Nginx Proxy Manager:**
1. ไปที่ Proxy Host → Edit
2. Tab "Advanced"
3. เพิ่ม Dozzle config จาก `complete-example.conf`
## 🔍 Features
### **1. Multi-Host Monitoring**
- ✅ ดู logs จาก Main Server
- ✅ ดู logs จาก Airbyte Host (192.168.100.9:7007)
- ✅ ดู logs จาก Airflow Host (192.168.100.9:7008)
- ✅ Switch ระหว่าง hosts ผ่าน dropdown
### **2. Real-time Log Streaming**
- Live log updates
- Color-coded logs
- JSON formatting
- Multi-line grouping
### **3. Container Management**
- View container stats (CPU, Memory, Network)
- Start/Stop/Restart containers
- Interactive shell access
- Container filtering
### **4. Advanced Features**
- Search และ filter logs
- Download logs
- Multiple container view
- SQL-based log querying
## 🐛 Troubleshooting
### **Issue: Agent ไม่ปรากฏใน UI**
**ตรวจสอบ:**
```bash
# 1. Agent ทำงานหรือไม่
docker ps | grep dozzle-agent
# 2. Port เปิดหรือไม่
netstat -tulpn | grep 7007
netstat -tulpn | grep 7008
# 3. Firewall
sudo ufw status
sudo ufw allow 7007
sudo ufw allow 7008
# 4. Network connectivity
ping 192.168.100.9
telnet 192.168.100.9 7007
telnet 192.168.100.9 7008
```
### **Issue: Connection Refused**
**สาเหตุ:**
- Agent ไม่ทำงาน
- Firewall block port
- Network ไม่เชื่อมต่อ
**วิธีแก้:**
```bash
# Restart agent
docker restart dozzle-agent-airbyte
docker restart dozzle-agent-airflow
# ตรวจสอบ logs
docker logs dozzle-agent-airbyte
docker logs dozzle-agent-airflow
# ทดสอบ connectivity
curl http://192.168.100.9:7007/healthcheck
curl http://192.168.100.9:7008/healthcheck
```
### **Issue: Containers ไม่แสดงใน Agent**
**สาเหตุ:**
- Docker socket ไม่ mount
- Agent ไม่มี permission
**วิธีแก้:**
```bash
# ตรวจสอบ volume mount
docker inspect dozzle-agent-airbyte | grep docker.sock
# ตรวจสอบ permissions
ls -la /var/run/docker.sock
# Restart agent
docker restart dozzle-agent-airbyte
```
## 🔐 Security Considerations
### **1. Network Security**
**ใช้ Internal Network (แนะนำ):**
```yaml
# Agent ไม่ expose port ออกภายนอก
# ใช้ Docker network แทน
dozzle-agent:
# ไม่ต้องมี ports section
networks:
- shared_data_network
```
**Main UI เชื่อมต่อผ่าน network:**
```yaml
DOZZLE_REMOTE_AGENT=dozzle-agent-airbyte:7007,dozzle-agent-airflow:7007
```
### **2. Firewall Rules**
```bash
# อนุญาตเฉพาะ Main Server
sudo ufw allow from <main-server-ip> to any port 7007
sudo ufw allow from <main-server-ip> to any port 7008
```
### **3. Authentication**
**Enable simple auth:**
```yaml
DOZZLE_AUTH_PROVIDER: simple
```
สร้าง `01-infra/data/dozzle/users.yml`:
```yaml
users:
- name: admin
username: admin
password: $2a$10$...
email: admin@sriphat.com
```
### **4. Read-only Docker Socket**
```yaml
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
```
## 📊 Monitoring
### **Health Checks**
```bash
# Main UI
curl http://localhost:9999/dozzle/healthcheck
# Airbyte Agent
curl http://192.168.100.9:7007/healthcheck
# Airflow Agent
curl http://192.168.100.9:7008/healthcheck
```
### **Logs**
```bash
# Main UI
docker logs dozzle -f
# Agents
docker logs dozzle-agent-airbyte -f
docker logs dozzle-agent-airflow -f
```
## 🎯 Best Practices
1. **ใช้ Internal Network** - ไม่ expose agent ports ออกภายนอก
2. **Enable Authentication** - ใช้ simple auth หรือ forward proxy
3. **Monitor Agent Health** - ตั้ง healthcheck และ alerting
4. **Backup Configuration** - backup `users.yml` และ `.env` files
5. **Update Regularly** - อัพเดท Dozzle image เป็นประจำ
6. **Use HTTPS** - ใช้ SSL/TLS สำหรับ production
7. **Limit Access** - ใช้ firewall และ access lists
## 📚 References
- [Dozzle Documentation](https://dozzle.dev/)
- [Agent Mode Guide](https://dozzle.dev/guide/agent)
- [Authentication Guide](https://dozzle.dev/guide/authentication)
- [Remote Hosts Guide](https://dozzle.dev/guide/remote-hosts)
## 🔄 Maintenance
### **Update Dozzle**
```bash
# Main UI
cd 01-infra
docker compose pull dozzle
docker compose up -d dozzle
# Agents
docker pull amir20/dozzle:latest
docker restart dozzle-agent-airbyte
docker restart dozzle-agent-airflow
```
### **Backup Configuration**
```bash
# Backup .env
cp .env.global .env.global.backup
# Backup users.yml (if using auth)
cp 01-infra/data/dozzle/users.yml users.yml.backup
```
## 🎉 Summary
**ตอนนี้คุณมี:**
- ✅ Dozzle Main UI บน Main Server
- ✅ Dozzle Agent บน Airbyte Host (192.168.100.9:7007)
- ✅ Dozzle Agent บน Airflow Host (192.168.100.9:7008)
- ✅ Nginx reverse proxy สำหรับ `/dozzle` subpath
- ✅ Multi-host monitoring ผ่าน single UI
- ✅ Real-time log streaming จากทุก hosts
**เข้าใช้งานที่:**
```
http://ai.sriphat.com/dozzle
```
**Features:**
- Monitor logs จาก Main Server, Airbyte, และ Airflow
- Real-time streaming
- Container stats
- Interactive shell
- Search และ filter

View File

@@ -0,0 +1,112 @@
---
tags:
- project/sriphat
- dataplatform
- infrastructure
created: 2026-05-07
status: active
project: 2026-SRI-PJ-001
---
# Sriphat Data Platform — Project Overview
## ข้อมูลโครงการ
| รายการ | รายละเอียด |
|--------|-----------|
| **โครงการ** | Sriphat AI Transformation Data Platform |
| **รหัสโครงการ** | 2026-SRI-PJ-001 |
| **องค์กร** | โรงพยาบาลศรีพัฒน์ (Sriphat Hospital) |
| **Domain** | `ai.sriphat.com` / `sriphat.local` |
| **Server IP** | `192.168.100.9` |
| **Timezone** | Asia/Bangkok (UTC+7) |
## วัตถุประสงค์
สร้างระบบ **Modern Data Stack** สำหรับโรงพยาบาลศรีพัฒน์ โดยเน้น:
- **Security** — ระบบยืนยันตัวตนกลาง (SSO) ผ่าน Keycloak
- **Versatility** — รองรับข้อมูลหลายรูปแบบ (SQL Server, Oracle, REST API, Excel)
- **Single Sign-On** — ผู้ใช้ล็อกอินครั้งเดียวเข้าได้ทุก service
## Architecture Overview
```
┌──────────────────────────────────────────────────────────┐
│ Nginx Reverse Proxy │
│ (Gateway + SSL + Subpath Routing) │
│ ai.sriphat.com │
└──────────────────────────────────────────────────────────┘
│ │ │
┌──────▼──────┐ ┌──────▼──────┐ ┌────▼────────┐
│ Keycloak │ │ API Service │ │ Superset │
│ (SSO) │ │ (FastAPI) │ │ (BI) │
│ /keycloak │ │ /apiservice │ │ /superset │
└─────────────┘ └─────────────┘ └─────────────┘
│ │ │
└────────────────┼───────────────┘
┌─────────────┼──────────────┐
│ │ │
┌─────▼────┐ ┌─────▼─────┐ ┌────▼─────┐
│PostgreSQL│ │ Supabase │ │ MinIO │
│(Infra DB)│ │(BaaS/API) │ │(S3 Store)│
└──────────┘ └───────────┘ └──────────┘
┌──────▼──────┐
│ Airflow │
│ (Workflow) │
└─────────────┘
```
## Tech Stack (Layer Map)
| Layer | Tool | หน้าที่ |
|-------|------|--------|
| **Gateway** | Nginx | จัดการ Domain, SSL, Subpath routing |
| **Identity (SSO)** | Keycloak 23.0 | ยืนยันตัวตนกลาง (OIDC/OAuth2), รองรับ LDAP/AD |
| **Backend API** | FastAPI (Python) | Custom API endpoints, API Key management |
| **Database (Infra)** | PostgreSQL 15 | ฐานข้อมูลหลักสำหรับ Keycloak, API Service |
| **BaaS** | Supabase | PostgreSQL + Auth + Realtime + Storage + Edge Functions |
| **Workflow** | Apache Airflow 3.1.5 | DAG-based workflow orchestration (CeleryExecutor) |
| **Ingestion** | Airbyte | ETL จาก HIS, Oracle, REST API (ปัจจุบัน commented out) |
| **Analytics** | Apache Superset | Business Intelligence Dashboard |
| **Object Storage** | MinIO | S3-compatible storage, รองรับ ML/AI workflows |
| **Cache/Queue** | Redis 7.2 | Celery broker สำหรับ Airflow |
| **Monitoring** | Dozzle | Docker container log viewer |
## Docker Network
ทุก service ใช้ network ร่วมกันชื่อ `shared_data_network` (external)
```bash
docker network create shared_data_network
```
## Service Ports (Quick Reference)
| Service | Container Port | Host Port | URL |
|---------|---------------|-----------|-----|
| Nginx Proxy | 80 | 8020 | `http://localhost:8020` |
| Keycloak | 8080 | 8085 | `http://localhost:8085/keycloak` |
| PostgreSQL (Infra) | 5432 | 5435 | internal |
| Supabase Studio | 3000 | 3010 | `http://localhost:3010` |
| Supabase Kong API | 8000 | 8100 | `http://localhost:8100` |
| Supabase DB | 5432 | 5434 | internal |
| Supabase Pooler | 6543 | 6544 | internal |
| API Service | 8040 | 8040 | `http://localhost:8040/apiservice` |
| Airflow API Server | 8080 | 8200 | `http://localhost:8200` |
| Superset | 8088 | 8088 | `http://localhost:8088` |
| MinIO API | 9000 | 9000 | `http://localhost:9000` |
| MinIO Console | 9001 | 9001 | `http://localhost:9001` |
| Dozzle | 8080 | 9999 | `http://localhost:9999/dozzle` |
## Related Documents
- [[01-Infrastructure]] — Nginx, Keycloak, PostgreSQL, Redis, Dozzle
- [[02-Supabase]] — BaaS layer (PostgreSQL + Auth + Realtime + Storage)
- [[03-API-Service]] — FastAPI custom endpoints
- [[04-Airflow]] — Workflow orchestration
- [[05-Analytics-Superset]] — BI Dashboard
- [[06-MinIO]] — Object Storage
- [[07-Security-Strategy]] — Security model และ SSO
- [[08-Operations-Runbook]] — Deploy, Backup, Troubleshoot

View File

@@ -0,0 +1,202 @@
---
tags:
- project/sriphat
- infrastructure
- nginx
- keycloak
- postgresql
created: 2026-05-07
status: active
folder: 01-infra
---
# Infrastructure Layer (01-infra)
> **Docker Compose:** `01-infra/docker-compose.yml`
> **Env File:** `.env.global`
## Services ใน Layer นี้
| Container | Image | Port | หน้าที่ |
|-----------|-------|------|--------|
| `nginx-proxy-manager` | nginx:latest | `8020:80` | Reverse proxy + Subpath routing |
| `keycloak` | quay.io/keycloak/keycloak:23.0 | `8085:8080` | SSO / Identity Provider |
| `postgres` | postgres:15-alpine | `5435:5432` | ฐานข้อมูลหลัก (Keycloak + API Service) |
| `redis` | redis:7.2-bookworm | internal | Cache / Message broker สำหรับ Airflow |
| `dozzle` | amir20/dozzle:latest | `9999:8080` | Docker log monitoring |
---
## Nginx Proxy Manager
**Image:** `nginx:latest`
### Subpath Routing Table
| Service | Subpath | Backend |
|---------|---------|---------|
| API Service | `/apiservice` | `apiservice:8040` |
| Supabase Studio | `/supabase` | `sdp-studio:3000` |
| Supabase Kong API | `/supabase-api` | `sdp-kong:8000` |
| Keycloak | `/keycloak` | `keycloak:8080` |
| Superset | `/superset` | `superset:8088` |
| Airflow | `/airflow` | `airflow-apiserver:8080` |
| Dozzle | `/dozzle` | `dozzle:8080` |
| MinIO API | `/minio` | `minio:9000` |
| MinIO Console | `/minio-console` | `minio:9001` |
**Config directory:** `01-infra/nginx-configs/`
### การตั้งค่า Nginx
```nginx
# เพิ่ม config ผ่าน Custom Nginx Configuration ใน Proxy Host
# หรือ mount file ไปที่ /etc/nginx/conf.d/default.conf
```
---
## Keycloak (SSO)
**Image:** `quay.io/keycloak/keycloak:23.0`
**URL:** `http://localhost:8085/keycloak` หรือ `https://ai.sriphat.com/keycloak`
### Configuration
```yaml
KC_DB: postgres
KC_DB_URL: jdbc:postgresql://postgres:5432/${KEYCLOAK_DB_NAME}
KC_HTTP_RELATIVE_PATH: "/keycloak"
KC_HOSTNAME_PATH: "/keycloak"
KC_PROXY: edge
KC_HTTP_ENABLED: "true"
```
### Keycloak SSO Clients ที่ต้องสร้าง
| Client ID | Service | Protocol |
|-----------|---------|---------|
| `apiservice` | API Service | OIDC |
| `superset-client` | Apache Superset | OIDC |
| `minio-client` | MinIO | OIDC |
| `airflow-client` | Apache Airflow | OIDC |
### ขั้นตอนตั้งค่า Keycloak หลัง Deploy
1. เข้า Admin Console: `/keycloak/admin`
2. สร้าง Realm: `sriphat`
3. สร้าง Clients สำหรับแต่ละ service
4. เชื่อมต่อ LDAP/AD ของโรงพยาบาล (optional)
5. สร้าง Groups และ Roles
6. Map roles ให้กับ users
---
## PostgreSQL (Infra DB)
**Image:** `postgres:15-alpine`
**Port:** `5435` (host) → `5432` (container)
### Databases ใน PostgreSQL นี้
| Database | เจ้าของ |
|----------|--------|
| `postgres` | Default + API Service |
| `keycloak` | Keycloak |
| `superset` | Apache Superset |
| `airflow` | Apache Airflow |
### Init Scripts
**Path:** `01-infra/init/`
| File | หน้าที่ |
|------|--------|
| `00-create-keycloak-database.sql` | สร้าง database สำหรับ Keycloak |
| `03-create-airflow-databases.sql` | สร้าง database สำหรับ Airflow |
### Connection String
```
Host: postgres (internal) / 192.168.100.9 (external)
Port: 5432 (internal) / 5435 (external)
User: ${DB_USER}
Password: ${DB_PASSWORD}
Database: postgres
```
---
## Redis
**Image:** `redis:7.2-bookworm`
**Port:** `6379` (internal only)
ใช้เป็น:
- Celery broker สำหรับ Apache Airflow
- Message queue
```
URL: redis://:@redis:6379/0
```
---
## Dozzle (Log Monitoring)
**Image:** `amir20/dozzle:latest`
**URL:** `http://localhost:9999/dozzle` หรือ `https://ai.sriphat.com/dozzle`
### Features
- ดู Docker container logs แบบ real-time
- รองรับ Remote Agent (เชื่อมต่อ server อื่น)
- Filter และ search logs
### Remote Agent Configuration
```bash
# ใน .env.global
DOZZLE_REMOTE_AGENT=192.168.100.9:7007
```
Server ที่ monitor:
- Main server (local)
- `192.168.100.9` — Airflow + MinIO + OpenMetadata server
**Setup Guide:** `REMOTE_HOSTS_DOZZLE_SETUP.md` — คู่มือตั้งค่า Dozzle agent บน remote server
---
## Environment Variables (.env.global)
```bash
# Project
PROJECT_NAME=sriphat-data
DOMAIN=sriphat.local
TZ=Asia/Bangkok
# Database
DB_HOST=postgres
DB_PORT=5432
DB_PORT_EXPOSE=5435
DB_USER=postgres
DB_PASSWORD=<secret>
DB_NAME=postgres
# Keycloak
KEYCLOAK_ADMIN=admin
KEYCLOAK_ADMIN_PASSWORD=<secret>
KEYCLOAK_DB_NAME=keycloak
# Dozzle
DOZZLE_PORT=9999
DOZZLE_BASE=/dozzle
DOZZLE_HOSTNAME=Sriphat Main Server
DOZZLE_REMOTE_AGENT=192.168.100.9:7007
```
## Related
- [[00-Project-Overview]]
- [[07-Security-Strategy]]
- [[08-Operations-Runbook]]

View File

@@ -0,0 +1,232 @@
---
tags:
- project/sriphat
- supabase
- postgresql
- baas
created: 2026-05-07
status: active
folder: 02-supabase
---
# Supabase Layer (02-supabase)
> **Docker Compose:** `02-supabase/docker-compose.yml`
> **Env File:** `02-supabase/.env`
Supabase เป็น Backend-as-a-Service (BaaS) แบบ self-hosted ที่รวม PostgreSQL, Auth, Realtime, Storage และ Edge Functions ไว้ในที่เดียว
## Services
| Container | Image | Port | หน้าที่ |
|-----------|-------|------|--------|
| `sdp-supabase-studio` | supabase/studio:2026.02.16 | `3010:3000` | Web UI สำหรับจัดการ database |
| `sdp-supabase-kong` | kong:2.8.1 | `8100:8000`, `8444:8443` | API Gateway (routing ทุก request) |
| `sdp-supabase-auth` | supabase/gotrue:v2.186.0 | internal | Authentication service |
| `sdp-supabase-rest` | postgrest/postgrest:v12.2.3 | internal | Auto-generated REST API จาก PostgreSQL |
| `sdp-realtime-dev` | supabase/realtime:v2.76.5 | internal | WebSocket realtime subscriptions |
| `sdp-supabase-storage` | supabase/storage-api:v1.37.8 | internal | File storage |
| `sdp-supabase-imgproxy` | darthsim/imgproxy:v3.30.1 | internal | Image transformation |
| `sdp-supabase-meta` | supabase/postgres-meta:v0.95.2 | internal | PostgreSQL metadata API |
| `sdp-supabase-edge-functions` | supabase/edge-runtime:v1.70.3 | internal | Deno edge functions |
| `sdp-supabase-analytics` | supabase/logflare:1.31.2 | internal | Log analytics (Logflare) |
| `sdp-supabase-db` | supabase/postgres:15.8.1.085 | `5434:5432` | PostgreSQL database หลัก |
| `sdp-supabase-vector` | timberio/vector:0.53.0-alpine | internal | Log collector |
| `sdp-supabase-pooler` | supabase/supavisor:2.7.4 | `6544:6543` | Connection pooler (PgBouncer-like) |
---
## สถาปัตยกรรม Supabase
```
Client / API Service
sdp-supabase-kong (API Gateway: port 8100)
┌────┼────────────────────┐
│ │ │
▼ ▼ ▼
Auth REST API Realtime
GoTrue PostgREST Supabase Realtime
│ │ │
└────┴────────────────────┘
sdp-supabase-db (PostgreSQL 15)
sdp-supabase-pooler
(Supavisor: port 6544)
```
---
## PostgreSQL Database (sdp-supabase-db)
**Image:** `supabase/postgres:15.8.1.085`
**Port:** `5434` (host)
### Init SQL Files
| File | หน้าที่ |
|------|--------|
| `volumes/db/realtime.sql` | Setup replication สำหรับ Realtime |
| `volumes/db/webhooks.sql` | Database webhook functions |
| `volumes/db/roles.sql` | PostgreSQL roles setup |
| `volumes/db/jwt.sql` | JWT helper functions |
| `volumes/db/_supabase.sql` | Internal Supabase schema |
| `volumes/db/logs.sql` | Logging tables |
| `volumes/db/pooler.sql` | Connection pooler config |
### Connection Strings
```
# Direct Connection
postgresql://postgres:<password>@sdp-supabase-db:5432/postgres
# Via Pooler (Transaction mode)
postgresql://postgres.tenant:<password>@sdp-supabase-pooler:6543/postgres
# External (from host)
postgresql://postgres:<password>@192.168.100.9:5434/postgres
```
---
## Kong API Gateway
**Image:** `kong:2.8.1`
**Port:** `8100` (HTTP), `8444` (HTTPS)
Kong ทำหน้าที่ route requests ไปยัง services ต่างๆ:
```
/auth/v1/* → sdp-supabase-auth (GoTrue)
/rest/v1/* → sdp-supabase-rest (PostgREST)
/realtime/v1/ → sdp-realtime (WebSocket)
/storage/v1/* → sdp-supabase-storage
/functions/v1/*→ sdp-supabase-edge-functions
/meta/* → sdp-supabase-meta
```
**Kong Config:** `volumes/api/kong.yml`
---
## Supavisor (Connection Pooler)
**Image:** `supabase/supavisor:2.7.4`
**Port:** `6544` (transaction mode pooler)
```bash
# Transaction mode (ใช้สำหรับ serverless/edge functions)
postgresql://postgres.sriphat:<password>@sdp-supabase-pooler:6543/postgres
POOLER_TENANT_ID: sriphat
POOLER_DEFAULT_POOL_SIZE: <from env>
POOLER_MAX_CLIENT_CONN: <from env>
```
---
## Authentication (GoTrue)
**Image:** `supabase/gotrue:v2.186.0`
### Features ที่เปิดใช้
| Feature | ค่า |
|---------|-----|
| Email Signup | `${ENABLE_EMAIL_SIGNUP}` |
| Anonymous Users | `${ENABLE_ANONYMOUS_USERS}` |
| Email Autoconfirm | `${ENABLE_EMAIL_AUTOCONFIRM}` |
| Phone Signup | `${ENABLE_PHONE_SIGNUP}` |
### JWT Configuration
```
JWT_SECRET: <จาก env>
JWT_EXPIRY: <จาก env>
JWT_AUD: authenticated
JWT_DEFAULT_GROUP: authenticated
```
---
## Storage
**Image:** `supabase/storage-api:v1.37.8`
**Data path:** `volumes/storage/`
```
FILE_SIZE_LIMIT: 52428800 (50MB)
STORAGE_BACKEND: file
ENABLE_IMAGE_TRANSFORMATION: true
```
---
## Supabase Studio
**URL:** `http://localhost:3010` หรือ `https://ai.sriphat.com/supabase`
**Image:** `supabase/studio:2026.02.16-sha-26c615c`
Studio เชื่อมต่อผ่าน:
- PostgreSQL Meta API (`sdp-meta:8080`)
- Kong API (`sdp-kong:8000`)
- Logflare (`sdp-analytics:4000`)
**Snippets path:** `volumes/snippets/`
**Functions path:** `volumes/functions/`
---
## Environment Variables (สำคัญ)
```bash
# PostgreSQL
POSTGRES_HOST=sdp-supabase-db
POSTGRES_PORT=5432
POSTGRES_DB=postgres
POSTGRES_PASSWORD=<secret>
# JWT
JWT_SECRET=<long-random-string>
JWT_EXPIRY=3600
# API Keys
ANON_KEY=<jwt-anon-key>
SERVICE_ROLE_KEY=<jwt-service-role-key>
# Studio
STUDIO_DEFAULT_ORGANIZATION=Sriphat Hospital
STUDIO_DEFAULT_PROJECT=DataPlatform
SUPABASE_PUBLIC_URL=https://ai.sriphat.com/supabase-api
# Logflare
LOGFLARE_PUBLIC_ACCESS_TOKEN=<token>
LOGFLARE_PRIVATE_ACCESS_TOKEN=<token>
```
---
## การใช้งาน Supabase จาก API Service
```python
# ใน 03-apiservice
SUPABASE_DB_HOST=sdp-supabase-db
SUPABASE_DB_PORT=5432
SUPABASE_DB_USER=postgres.1
SUPABASE_DB_NAME=postgres
SUPABASE_API_URL=http://sdp-kong:8000
SUPABASE_API_KEY=<anon-or-service-role-key>
```
---
## Related
- [[00-Project-Overview]]
- [[03-API-Service]]
- [[07-Security-Strategy]]

View File

@@ -0,0 +1,251 @@
---
tags:
- project/sriphat
- apiservice
- fastapi
- python
created: 2026-05-07
status: active
folder: 03-apiservice
---
# API Service (03-apiservice)
> **Docker Compose:** `03-apiservice/docker-compose.yml`
> **Env File:** `03-apiservice/.env`
> **Language:** Python / FastAPI
## Overview
Custom FastAPI service สำหรับ:
- รับข้อมูล Checkpoint จาก HIS (Hospital Information System)
- จัดการ API Keys แบบ permission-based
- Admin UI สำหรับบริหาร API Clients
- รองรับ Keycloak SSO สำหรับหน้าเว็บ Admin
## Container
| รายการ | ค่า |
|--------|-----|
| **Container** | `apiservice` |
| **Image** | `03-apiservice-apiservice:latest` (build local) |
| **Port** | `8040:8040` |
| **URL** | `https://ai.sriphat.com/apiservice` |
| **Health Check** | `http://localhost:8040/apiservice/docs` |
---
## API Endpoints (หลัก)
### Data Feed Endpoints
```
POST /apiservice/api/v1/feed/checkpoint
```
**Payload ตัวอย่าง:**
```json
[
{
"id": 1,
"hn": 123,
"vn": 456,
"location": "OPD",
"type": "Scan",
"timestamp_in": "2026-02-16T10:00:00",
"timestamp_out": null,
"waiting_time": null,
"bu": "SRIPHAT"
}
]
```
**Required Permission:** `feed.checkpoint:write`
### Admin Endpoints
```
GET /apiservice/admin/ # Admin dashboard
POST /apiservice/admin/api-keys/generate # สร้าง API Key ใหม่
GET /apiservice/admin/api-clients # รายการ API Clients
```
### Documentation
```
GET /apiservice/docs # Swagger UI
GET /apiservice/redoc # ReDoc
```
---
## Database Schema
API Service ใช้ PostgreSQL (Infra) และ Supabase:
### Tables (PostgreSQL Infra)
| Table | ใช้สำหรับ |
|-------|---------|
| `fastapi.ApiClient` | ข้อมูล API Client (ระบบที่ขอใช้ API) |
| `fastapi.ApiKey` | API Keys ที่เข้ารหัสแล้ว |
### Tables (Supabase)
| Table | Schema | ใช้สำหรับ |
|-------|--------|---------|
| `RawWaitingTime` | `operationbi` | ข้อมูล waiting time ดิบ |
| `RawOpdCheckpoint` | — | ข้อมูล OPD checkpoint |
---
## Authentication
### 1. API Key Authentication (สำหรับ System Integration)
```bash
# Request header
Authorization: Bearer <api-key>
# หรือ query param
?api_key=<api-key>
```
API Key สร้างได้จาก Admin UI โดยกำหนด permissions:
- `feed.checkpoint:write` — บันทึกข้อมูล checkpoint
- (สามารถเพิ่ม permissions เพิ่มเติมได้)
### 2. Keycloak SSO (สำหรับ Admin Web UI)
```bash
# Environment variables
KEYCLOAK_SERVER_URL=http://keycloak:8080
KEYCLOAK_REALM=master
KEYCLOAK_CLIENT_ID=apiservice
KEYCLOAK_CLIENT_SECRET=<secret>
KEYCLOAK_REDIRECT_URI=http://localhost:8040/apiservice/auth/callback
```
---
## File Structure
```
03-apiservice/
├── app/
│ ├── api/v1/
│ │ ├── routes.py # API endpoints
│ │ └── schemas.py # Pydantic schemas
│ ├── core/
│ │ └── config.py # Settings / Config
│ ├── db/
│ │ ├── models.py # SQLAlchemy models
│ │ ├── init_db.py # Database initialization
│ │ └── session.py # DB session
│ ├── middleware/ # Custom middleware
│ ├── models/ # Additional models
│ ├── routes/ # Additional routes
│ ├── security/
│ │ ├── api_key.py # API Key handling
│ │ ├── keycloak_auth.py # Keycloak integration
│ │ ├── permissions.py # Permission system
│ │ └── dependencies.py # FastAPI dependencies
│ ├── services/ # Business logic
│ ├── templates/ # HTML templates (Admin UI)
│ └── utils/
│ └── supabase_client.py
├── data/uploads/ # File uploads
├── docker-compose.yml
├── requirements.txt
└── .env
```
---
## Environment Variables
```bash
# Application
APP_NAME=APIsService
ROOT_PATH=/apiservice
TIMEZONE=Asia/Bangkok
# PostgreSQL (Infra DB)
DB_HOST=postgres
DB_PORT=5432
DB_USER=postgres
DB_PASSWORD=<secret>
DB_NAME=postgres
DB_SSLMODE=prefer
# Supabase DB (สำหรับ RawOpdCheckpoint)
SUPABASE_DB_HOST=sdp-supabase-db
SUPABASE_DB_PORT=5432
SUPABASE_DB_USER=postgres.1
SUPABASE_DB_NAME=postgres
# Supabase API
SUPABASE_API_URL=http://sdp-kong:8000
SUPABASE_API_KEY=<anon-or-service-role-key>
# Admin
ADMIN_SECRET_KEY=<secret>
ADMIN_USERNAME=admin
ADMIN_PASSWORD=<secret>
API_KEY_ENC_SECRET=<encryption-key>
# Keycloak
KEYCLOAK_SERVER_URL=http://keycloak:8080
KEYCLOAK_REALM=master
KEYCLOAK_CLIENT_ID=apiservice
KEYCLOAK_CLIENT_SECRET=<secret>
KEYCLOAK_REDIRECT_URI=<redirect-url>
# Airflow Integration
AIRFLOW_API_URL=http://airflow-webserver:8080
AIRFLOW_API_TOKEN=<token>
AIRFLOW_DAG_ID_FINANCE=process_finance_excel
# Debug
DEBUG_AUTH=false
LOG_LEVEL=debug
```
---
## Build & Deploy
```bash
# Build image
cd 03-apiservice
docker compose --env-file ../.env.global build
# Start service
docker compose --env-file ../.env.global up -d
# View logs
docker logs apiservice -f
# Restart
docker restart apiservice
```
---
## Airflow Integration
API Service มี integration กับ Apache Airflow:
- ส่ง trigger ไปยัง Airflow DAG
- DAG `process_finance_excel` สำหรับประมวลผล Excel files
ดูรายละเอียดที่ `03-apiservice/AIRFLOW_INTEGRATION.md`
---
## Related
- [[00-Project-Overview]]
- [[01-Infrastructure]]
- [[02-Supabase]]
- [[04-Airflow]]
- [[07-Security-Strategy]]

View File

@@ -0,0 +1,237 @@
---
tags:
- project/sriphat
- airflow
- workflow
- etl
created: 2026-05-07
status: active
folder: 05-airflow
---
# Apache Airflow (05-airflow)
> **Docker Compose:** `05-airflow/docker-compose.yaml`
> **Env File:** `05-airflow/.env`
> **Version:** Apache Airflow 3.1.5
## Overview
Apache Airflow ใช้สำหรับ Workflow Orchestration:
- รัน DAGs (Directed Acyclic Graphs) แบบตั้งเวลา
- ประมวลผล Excel/CSV files จาก Finance
- ETL pipeline orchestration
- Integration กับ API Service
**Executor:** CeleryExecutor (ใช้ Redis เป็น broker)
---
## Services
| Container | หน้าที่ | Port |
|-----------|--------|------|
| `airflow-apiserver` | REST API + Web UI | `8200:8080` |
| `airflow-scheduler` | DAG scheduling | internal |
| `airflow-dag-processor` | DAG file parsing | internal |
| `airflow-worker` | Task execution (Celery) | internal |
| `airflow-triggerer` | Deferred task triggering | internal |
| `airflow-init` | Database migration (one-time) | — |
| `airflow-cli` | CLI tool (debug profile) | — |
| `flower` | Celery monitoring (optional) | `5555:5555` |
---
## Architecture
```
┌─────────────────┐
│ airflow- │
│ apiserver │ ← Web UI + REST API (port 8200)
│ (port 8080) │
└────────┬────────┘
┌───────────────┼───────────────┐
│ │ │
┌──────▼──────┐ ┌──────▼──────┐ ┌─────▼──────┐
│ airflow- │ │ airflow- │ │ airflow- │
│ scheduler │ │ dag- │ │ triggerer │
│ │ │ processor │ │ │
└──────┬──────┘ └─────────────┘ └────────────┘
▼ (Celery tasks via Redis)
┌──────────────┐
│ airflow- │
│ worker │ ← รัน tasks จริง
└──────────────┘
┌──────────────┐
│ PostgreSQL │ (Airflow metadata DB)
│ Redis │ (Celery broker)
└──────────────┘
```
---
## Database Configuration
Airflow ใช้ PostgreSQL บน Infra server:
```bash
# Connection string
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN=
postgresql+psycopg2://${AIRFLOW_DB_USER}:${AIRFLOW_DB_PASSWD}@${AIRFLOW_DB_HOST}:${AIRFLOW_DB_PORT}/${AIRFLOW_DB_NAME}
AIRFLOW__CELERY__RESULT_BACKEND=
db+postgresql://${AIRFLOW_DB_USER}:${AIRFLOW_DB_PASSWD}@${AIRFLOW_DB_HOST}:${AIRFLOW_DB_PORT}/${AIRFLOW_DB_NAME}
# Redis broker
AIRFLOW__CELERY__BROKER_URL=redis://:@redis:6379/0
```
---
## Volume Mounts
```
05-airflow/
├── dags/ → /opt/airflow/dags (DAG files)
├── logs/ → /opt/airflow/logs (Task logs)
├── config/ → /opt/airflow/config (airflow.cfg)
│ └── airflow.cfg
└── plugins/ → /opt/airflow/plugins (Custom plugins)
```
---
## Web UI
**URL:** `http://localhost:8200` หรือ `https://ai.sriphat.com/airflow`
```bash
# Config
AIRFLOW__WEBSERVER__BASE_URL=https://ai.sriphat.com/airflow
AIRFLOW__WEBSERVER__WEB_SERVER_PORT=8080
```
Default credentials (ถ้าไม่เปลี่ยน):
- Username: `airflow`
- Password: `airflow`
---
## DAGs ที่มีอยู่
| DAG ID | หน้าที่ | ถูก Trigger จาก |
|--------|--------|----------------|
| `process_finance_excel` | ประมวลผล Excel ของ Finance | API Service |
---
## Airflow Configuration (airflow.cfg)
**Path:** `05-airflow/config/airflow.cfg`
Key settings:
```ini
[core]
executor = CeleryExecutor
load_examples = False
dags_are_paused_at_creation = True
[webserver]
base_url = https://ai.sriphat.com/airflow
[execution_api]
execution_api_server_url = http://airflow-apiserver:8080/execution/
```
---
## Environment Variables
```bash
# Airflow image
AIRFLOW_IMAGE_NAME=apache/airflow:3.1.5
# Database
AIRFLOW_DB_USER=<user>
AIRFLOW_DB_PASSWD=<password>
AIRFLOW_DB_HOST=<postgres-host>
AIRFLOW_DB_PORT=5432
AIRFLOW_DB_NAME=airflow
# Security
AIRFLOW__CORE__FERNET_KEY=<fernet-key>
# Admin user
_AIRFLOW_WWW_USER_USERNAME=airflow
_AIRFLOW_WWW_USER_PASSWORD=<password>
# Optional pip packages
_PIP_ADDITIONAL_REQUIREMENTS=
```
---
## Deploy Commands
```bash
cd 05-airflow
# Initialize (first time only)
docker compose up airflow-init
# Start all services
docker compose up -d
# View logs
docker logs airflow-apiserver -f
docker logs airflow-scheduler -f
docker logs airflow-worker -f
# Run Celery Flower monitoring
docker compose --profile flower up -d
# Scale workers (เพิ่ม worker)
docker compose up -d --scale airflow-worker=3
```
---
## System Requirements
Airflow ต้องการ resources ขั้นต่ำ:
- **RAM:** ≥ 4 GB
- **CPU:** ≥ 2 cores
- **Disk:** ≥ 10 GB
---
## Ingestion Layer (04-ingestion / Airbyte)
> **หมายเหตุ:** `04-ingestion/docker-compose.yml` ปัจจุบัน **commented out ทั้งหมด**
> Airbyte ถูก deploy แยกต่างหาก (ผ่าน `abctl` หรือ standalone)
### Airbyte ที่ระบุในแผน
| Source | ชนิดข้อมูล |
|--------|----------|
| SQL Server (HIS) | ข้อมูลผู้ป่วย, OPD |
| Oracle (Lab) | ผลตรวจทางห้องปฏิบัติการ |
| REST API | External data |
| Excel/CSV | Finance, รายงาน |
**Destination:** PostgreSQL `raw_data` schema
**Port:** `8030` (เมื่อ deploy แล้ว)
---
## Related
- [[00-Project-Overview]]
- [[01-Infrastructure]]
- [[03-API-Service]]
- [[08-Operations-Runbook]]

View File

@@ -0,0 +1,243 @@
---
tags:
- project/sriphat
- superset
- analytics
- bi
- dashboard
created: 2026-05-07
status: active
folder: 06-analytics
---
# Apache Superset — Analytics Layer (06-analytics)
> **Docker Compose:** `06-analytics/docker-compose.yml`
> **Env File:** `.env` (global)
## Overview
Apache Superset ใช้เป็น Business Intelligence (BI) platform สำหรับ:
- สร้าง Dashboard และ Visualization
- เชื่อมต่อกับ PostgreSQL Data Warehouse
- **Embedded Superset SDK** — embed dashboard ใน applications อื่นโดยไม่ต้อง login
- สร้าง Report สำหรับผู้บริหารและแพทย์
> **SSO Keycloak:** ยังอยู่ในแผน ยังไม่ได้ implement
---
## Container
| รายการ | ค่า |
|--------|-----|
| **Container** | `superset` |
| **Image** | Build จาก `Dockerfile` ใน `06-analytics/` |
| **Port** | `8088:8088` |
| **URL** | `http://localhost:8088` หรือ `https://ai.sriphat.com/superset` |
| **Network** | `shared_data_network` |
---
## Configuration
### Database Connection (Superset metadata)
Superset เก็บ metadata ใน PostgreSQL (Infra):
```
Database: superset
Host: ${DB_HOST}
Port: 5432
User: ${DB_USER}
Password: ${DB_PASSWORD}
```
### Superset Config File
**Path:** `06-analytics/superset_config.py`
```python
SECRET_KEY = os.environ.get('SUPERSET_SECRET_KEY')
ENABLE_PROXY_FIX = True
PUBLIC_ROLE_LIKE = "Gamma"
GUEST_ROLE_NAME = "Gamma"
# CSRF
WTF_CSRF_ENABLED = False
WTF_CSRF_TIME_LIMIT = None
# CORS — อนุญาตทุก origin (ปรับ production ให้ restrictive กว่านี้)
ENABLE_CORS = True
CORS_OPTIONS = {
'supports_credentials': True,
'allow_headers': ['*'],
'resources': ['*'],
'origins': ['*']
}
SESSION_COOKIE_SAMESITE = "Lax"
SESSION_COOKIE_SECURE = False
# Embedded Superset SDK
FEATURE_FLAGS = {"EMBEDDED_SUPERSET": True}
EMBEDDED_SUPERSET = True
TALISMAN_ENABLED = False
ENABLE_TEMPLATE_PROCESSING = True
LOGO_TARGET_PATH = '/superset/welcome/'
# Guest Token (สำหรับ embedded dashboard ไม่ต้อง login)
GUEST_TOKEN_JWT_SECRET = '<secret>'
GUEST_TOKEN_JWT_EXP_SECONDS = 300 # 5 นาที
GUEST_TOKEN_JWT_ALGORITHM = "HS256"
# Domain whitelist สำหรับ embed
EMBEDDED_SDK_HOST_WHITELIST = [
"https://ai.sriphat.com",
"http://localhost:8800",
"http://127.0.0.1:8800"
]
```
> **หมายเหตุ:** Keycloak SSO ยังไม่ได้ integrate — ปัจจุบันใช้ Username/Password login + Embedded SDK สำหรับ embed dashboard ใน applications อื่น
### Environment Variables
```bash
SUPERSET_SECRET_KEY=${SUPERSET_SECRET_KEY}
DATABASE_DIALECT=postgresql
DATABASE_HOST=${DB_HOST}
DATABASE_PORT=5432
DATABASE_DB=superset
DATABASE_USER=${DB_USER}
DATABASE_PASSWORD=${DB_PASSWORD}
SUPERSET_LOAD_EXAMPLES=no
SUPERSET_BIND_ADDRESS=0.0.0.0
SUPERSET_PORT=8088
TZ=Asia/Bangkok
```
---
## Startup Process
เมื่อ container เริ่มทำงาน จะรันคำสั่งต่อไปนี้โดยอัตโนมัติ:
```bash
# 1. Migrate database
superset db upgrade
# 2. Create admin user
superset fab create-admin \
--username ${SUPERSET_ADMIN_USERNAME} \
--firstname Admin \
--lastname User \
--email admin@sriphat.local \
--password ${SUPERSET_ADMIN_PASSWORD}
# 3. Initialize Superset
superset init
# 4. Start Gunicorn server
gunicorn --bind 0.0.0.0:8088 \
--workers 4 \
--timeout 120 \
'superset.app:create_app()'
```
---
## Data Sources ที่ Connect ได้
### PostgreSQL Data Warehouse (Infra)
```
postgresql://postgres:<password>@postgres:5432/postgres
```
**Schemas ที่แนะนำให้ expose:**
- `analytics` — ข้อมูลที่ transform แล้ว (read-only สำหรับ BI)
- `operationbi` — ข้อมูล Operation BI
### Supabase PostgreSQL
```
postgresql://postgres:<password>@sdp-supabase-db:5432/postgres
```
---
## Dashboard ที่ควรสร้าง
| Dashboard | ข้อมูล | ผู้ใช้ |
|-----------|--------|--------|
| OPD Waiting Time | `RawWaitingTime`, `RawOpdCheckpoint` | ผู้บริหาร, พยาบาล |
| Patient Flow | HIS data จาก Airbyte | แพทย์, ผู้บริหาร |
| Finance Overview | Excel จาก Finance | CFO, ผู้บริหาร |
| Department KPIs | Aggregated data | หัวหน้าแผนก |
---
## Security (Row-Level Security)
ตั้งค่า RLS ใน Superset เพื่อจำกัดข้อมูล:
```sql
-- ตัวอย่าง: แพทย์เห็นเฉพาะผู้ป่วยของตัวเอง
-- ใน Superset: Security → Row Level Security → Add Rule
-- Filter: department = '{{current_username}}'
```
---
## Volume Mounts
```
06-analytics/
├── data/superset_home/ # Superset config + cache
└── superset_config.py # Custom configuration
```
---
## Build & Deploy
```bash
cd 06-analytics
# Build image (มี custom Dockerfile สำหรับเพิ่ม packages)
docker compose --env-file ../.env.global build
# Start
docker compose --env-file ../.env.global up -d
# View logs
docker logs superset -f
```
---
## Development Version (06-analytics-dev)
มี docker-compose สำหรับ development แยกต่างหาก:
- **Path:** `06-analytics-dev/docker-compose.yml`
- ใช้สำหรับทดสอบ config ก่อน deploy production
---
## Access
| รายการ | ค่า |
|--------|-----|
| URL | `http://localhost:8088` |
| Admin Username | ค่าจาก `SUPERSET_ADMIN_USERNAME` |
| Admin Password | ค่าจาก `SUPERSET_ADMIN_PASSWORD` |
---
## Related
- [[00-Project-Overview]]
- [[01-Infrastructure]] (Keycloak SSO)
- [[02-Supabase]] (Data Source)
- [[07-Security-Strategy]]

View File

@@ -0,0 +1,264 @@
---
tags:
- project/sriphat
- minio
- storage
- s3
created: 2026-05-07
status: active
folder: 07-minio
---
# MinIO Object Storage (07-minio)
> **Docker Compose:** `07-minio/docker-compose.yml`
> **Env File:** `07-minio/.env`
## Overview
MinIO เป็น S3-compatible object storage สำหรับ:
- เก็บ raw data files (CSV, JSON, Parquet)
- เก็บ ML/AI models และ training data
- เก็บ backups และ reports
- Keycloak SSO integration
---
## Container
| รายการ | ค่า |
|--------|-----|
| **Container** | `minio` |
| **Image** | `minio/minio:latest` |
| **API Port** | `9000:9000` |
| **Console Port** | `9001:9001` |
| **Console URL** | `https://ai.sriphat.com/minio-console` |
| **API URL** | `https://ai.sriphat.com/minio` |
| **Direct (Dev)** | `http://192.168.100.9:9001` (console) |
| **Region** | `ap-southeast-1` |
---
## Use Cases
| Use Case | ตัวอย่าง |
|----------|---------|
| **Data Lake** | Raw CSV, JSON, Parquet จาก Airbyte |
| **ML/AI Workflows** | Model files, training datasets, experiment artifacts |
| **Backup Storage** | Database dumps, application backups |
| **Report Files** | Excel, PDF reports จาก Finance |
| **Media Storage** | Images, documents จากระบบ HIS |
| **Application Storage** | File uploads จาก API Service |
---
## Authentication
### 1. Root Credentials (Default)
```bash
MINIO_ROOT_USER=minioadmin
MINIO_ROOT_PASSWORD=<strong-password>
```
### 2. Keycloak SSO (แนะนำ)
เชื่อมต่อผ่าน OpenID Connect:
```bash
MINIO_IDENTITY_OPENID_CONFIG_URL=https://ai.sriphat.com/keycloak/realms/sriphat/.well-known/openid-configuration
MINIO_IDENTITY_OPENID_CLIENT_ID=minio-client
MINIO_IDENTITY_OPENID_CLIENT_SECRET=<secret>
MINIO_IDENTITY_OPENID_CLAIM_NAME=policy
MINIO_IDENTITY_OPENID_SCOPES=openid,profile,email
MINIO_IDENTITY_OPENID_REDIRECT_URI=https://ai.sriphat.com/minio-console/oauth_callback
```
**Policy Mapping:** User ใน Keycloak ต้องมี attribute `policy` ที่ map กับ MinIO policy
---
## Environment Variables
```bash
# Credentials
MINIO_ROOT_USER=minioadmin
MINIO_ROOT_PASSWORD=<secret>
# URLs
MINIO_SERVER_URL=https://ai.sriphat.com/minio
MINIO_BROWSER_REDIRECT_URL=https://ai.sriphat.com/minio-console
# Region
MINIO_REGION=ap-southeast-1
# Keycloak SSO
MINIO_IDENTITY_OPENID_CONFIG_URL=<keycloak-oidc-url>
MINIO_IDENTITY_OPENID_CLIENT_ID=<client-id>
MINIO_IDENTITY_OPENID_CLIENT_SECRET=<secret>
MINIO_IDENTITY_OPENID_CLAIM_NAME=policy
MINIO_IDENTITY_OPENID_SCOPES=openid,profile,email
MINIO_IDENTITY_OPENID_REDIRECT_URI=<redirect-uri>
TZ=Asia/Bangkok
```
---
## Volume Mounts
```
07-minio/
├── data/ → /data (object storage data)
└── certs/ → /root/.minio/certs:ro (SSL certificates)
```
---
## การใช้งาน MinIO Client (mc)
```bash
# Install
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc && sudo mv mc /usr/local/bin/
# Config alias
mc alias set sriphat https://ai.sriphat.com/minio minioadmin <password>
# List buckets
mc ls sriphat
# Create bucket
mc mb sriphat/raw-data
mc mb sriphat/ml-models
mc mb sriphat/backups
mc mb sriphat/reports
# Upload
mc cp data.csv sriphat/raw-data/
mc cp -r ./models/ sriphat/ml-models/
# Set bucket policy
mc anonymous set none sriphat/raw-data # private
mc anonymous set download sriphat/public # public read
```
---
## Python SDK (boto3)
```python
import boto3
from botocore.client import Config
s3 = boto3.client(
's3',
endpoint_url='https://ai.sriphat.com/minio',
aws_access_key_id='minioadmin',
aws_secret_access_key='<password>',
config=Config(signature_version='s3v4'),
region_name='ap-southeast-1'
)
# Upload file
s3.upload_file('data.csv', 'raw-data', 'data.csv')
# Download file
s3.download_file('raw-data', 'data.csv', 'local-data.csv')
# List objects
for obj in s3.list_objects_v2(Bucket='raw-data').get('Contents', []):
print(obj['Key'])
```
---
## Recommended Bucket Structure
```
sriphat/
├── raw-data/ # ข้อมูลดิบจาก Airbyte / HIS
│ ├── his/
│ ├── oracle-lab/
│ └── finance-excel/
├── processed-data/ # ข้อมูลที่ transform แล้ว
├── ml-models/ # ML/AI model files
│ ├── waiting-time/
│ └── patient-flow/
├── reports/ # Excel, PDF reports
├── backups/ # Database backups
│ └── postgres/
└── uploads/ # User uploads จาก API Service
```
---
## Security
```bash
# สร้าง read-only policy
cat > readonly-policy.json << 'EOF'
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:ListBucket"],
"Resource": ["arn:aws:s3:::*"]
}
]
}
EOF
mc admin policy create sriphat readonly-policy readonly-policy.json
# Assign policy ให้ user
mc admin policy attach sriphat readonly-policy --user=analyst-user
```
---
## Health Check
```bash
# ตรวจสอบสถานะ
curl -f http://localhost:9000/minio/health/live
docker exec minio curl -f http://localhost:9000/minio/health/live
```
---
## Backup Strategy
```bash
# Backup data directory
tar -czf minio-backup-$(date +%Y%m%d).tar.gz 07-minio/data/
# Sync to remote
rsync -avz 07-minio/data/ backup-server:/backups/minio/
# Restore
docker compose down
tar -xzf minio-backup-20260501.tar.gz
docker compose up -d
```
---
## Keycloak Setup (สำหรับ SSO)
ดูรายละเอียดที่ `07-minio/KEYCLOAK_INTEGRATION.md`
1. สร้าง Client `minio-client` ใน Keycloak Realm `sriphat`
2. ตั้งค่า Valid Redirect URIs: `https://ai.sriphat.com/minio-console/oauth_callback`
3. สร้าง Client Scope `minio-policy`
4. เพิ่ม User Attribute Mapper `policy`
5. กำหนด `policy` attribute ให้กับ users ตาม MinIO policies
---
## Related
- [[00-Project-Overview]]
- [[01-Infrastructure]] (Keycloak SSO)
- [[07-Security-Strategy]]

View File

@@ -0,0 +1,217 @@
---
tags:
- project/sriphat
- security
- sso
- keycloak
- rbac
created: 2026-05-07
status: active
---
# Security Strategy
## หลักการออกแบบ
ระบบออกแบบตามหลัก **Defense in Depth** สำหรับสภาพแวดล้อมโรงพยาบาล:
1. **Network Isolation** — ทุก service อยู่ใน `shared_data_network`
2. **Centralized Authentication** — SSO ผ่าน Keycloak เท่านั้น
3. **Schema Separation** — แยก raw / analytics / production data
4. **Row-Level Security** — PostgreSQL RLS จำกัดข้อมูลต่อ user
5. **API Key Management** — Permission-based API access
---
## Network Security
```
Internet
Nginx Reverse Proxy (port 80/443)
│ ← ทุก request ต้องผ่าน Nginx
shared_data_network (Docker internal)
├── keycloak
├── postgres
├── sdp-supabase-* (12 containers)
├── apiservice
├── superset
├── airflow-*
├── minio
└── redis
```
**Rules:**
- ไม่มี service ใดที่ bind ตรงกับ `0.0.0.0` ยกเว้นผ่าน Nginx
- Redis ไม่ expose port ออกภายนอก
- Supabase services ไม่ expose port ออกภายนอก (ผ่าน Kong เท่านั้น)
---
## Authentication (SSO with Keycloak)
### Flow
```
User
├─ Web UI → Keycloak (OIDC Login) → redirect back with token
└─ API Client → API Key (จาก Admin UI) → Bearer token
```
### Keycloak Realms
| Realm | Services |
|-------|---------|
| `master` | Admin (ปิด public access) |
| `sriphat` | Superset, MinIO, Airflow, API Service |
### Client Configurations
| Service | Client ID | Flow |
|---------|-----------|------|
| API Service | `apiservice` | Authorization Code |
| Superset | `superset-client` | Authorization Code |
| MinIO | `minio-client` | Authorization Code + PKCE |
| Airflow | `airflow-client` | Authorization Code |
---
## API Key Security (API Service)
### Permission System
```
API Client (ระบบที่ขอใช้ API)
└── API Keys (ถูก encrypt ด้วย AES)
└── Permissions:
- feed.checkpoint:write
- (เพิ่มได้ตามต้องการ)
```
### API Key Lifecycle
```bash
# 1. Admin สร้าง API Client
POST /apiservice/admin/api-clients
# 2. Generate API Key สำหรับ Client
POST /apiservice/admin/api-keys/generate
?client_id=1
&permissions=feed.checkpoint:write
&name=his-production-key
# 3. ใช้งาน API Key
curl -H "Authorization: Bearer <key>" \
https://ai.sriphat.com/apiservice/api/v1/feed/checkpoint
```
---
## Database Security (Schema Separation)
### PostgreSQL Schemas
| Schema | ใช้งาน | สิทธิ์ |
|--------|--------|--------|
| `public` | Default | แล้วแต่ config |
| `raw_data` | ข้อมูลดิบจาก Airbyte | Airflow write, BI read-only |
| `analytics` | ข้อมูลที่ transform แล้ว | BI read-only |
| `operationbi` | Operation KPIs | API Service write, Superset read |
| `fastapi` | API Service metadata | API Service only |
| `_analytics` | Supabase Logflare | Internal only |
| `_realtime` | Supabase Realtime | Internal only |
### Row-Level Security (RLS)
```sql
-- ตัวอย่าง: จำกัดข้อมูลตาม department
ALTER TABLE patient_data ENABLE ROW LEVEL SECURITY;
CREATE POLICY department_isolation ON patient_data
USING (department = current_setting('app.current_department'));
-- Set context ใน connection
SET app.current_department = 'OPD';
```
---
## Secrets Management
### ไฟล์ Secrets
| ไฟล์ | ความสำคัญ | ต้องทำ |
|------|---------|--------|
| `.env.global` | HIGH | ไม่ commit ลง git |
| `02-supabase/.env` | HIGH | ไม่ commit ลง git |
| `03-apiservice/.env` | HIGH | ไม่ commit ลง git |
| `07-minio/.env` | HIGH | ไม่ commit ลง git |
### Key Secrets ที่ต้อง Rotate
| Secret | Location | แนะนำ Rotation |
|--------|----------|--------------|
| `DB_PASSWORD` | `.env.global` | 90 วัน |
| `JWT_SECRET` (Supabase) | `02-supabase/.env` | เมื่อมีเหตุ |
| `KEYCLOAK_ADMIN_PASSWORD` | `.env.global` | 90 วัน |
| `ADMIN_SECRET_KEY` (API) | `03-apiservice/.env` | 90 วัน |
| `MINIO_ROOT_PASSWORD` | `07-minio/.env` | 90 วัน |
| `AIRFLOW__CORE__FERNET_KEY` | `05-airflow/.env` | ไม่ rotate (data loss) |
| `SUPERSET_SECRET_KEY` | `.env.global` | ไม่ rotate (session loss) |
---
## Security Checklist
### Pre-Production
- [ ] เปลี่ยน passwords ทั้งหมดจาก default
- [ ] เปิด HTTPS ใน Nginx (Let's Encrypt หรือ internal CA)
- [ ] ตั้งค่า Keycloak realm `sriphat` (ไม่ใช่ `master`)
- [ ] เชื่อมต่อ LDAP/AD ของโรงพยาบาล
- [ ] Enable RLS ใน PostgreSQL
- [ ] ตั้งค่า firewall rules (จำกัด inbound ports)
- [ ] Setup audit logging
- [ ] กำหนด session timeout ใน Keycloak
### Ongoing
- [ ] Review API Keys ที่ active ทุก 30 วัน
- [ ] Monitor Dozzle สำหรับ unusual access patterns
- [ ] Backup secrets ไว้ใน secure vault (HashiCorp Vault หรือ similar)
- [ ] Rotate passwords ตามกำหนด
---
## SSL/TLS Configuration
Nginx จัดการ SSL termination:
```nginx
server {
listen 443 ssl;
server_name ai.sriphat.com;
ssl_certificate /etc/nginx/ssl/cert.pem;
ssl_certificate_key /etc/nginx/ssl/key.pem;
# Force HTTPS
# Redirect http → https ผ่าน Nginx
}
```
---
## Related
- [[00-Project-Overview]]
- [[01-Infrastructure]] (Keycloak setup)
- [[03-API-Service]] (API Key management)
- [[08-Operations-Runbook]]

View File

@@ -0,0 +1,401 @@
---
tags:
- project/sriphat
- operations
- runbook
- deployment
- troubleshooting
created: 2026-05-07
status: active
---
# Operations Runbook
## Quick Reference — Service Status
```bash
# ดู containers ทั้งหมดที่รันอยู่
docker ps
# ดู resource usage
docker stats
# ดู logs แบบ realtime (ผ่าน Dozzle)
# https://ai.sriphat.com/dozzle
```
---
## First-Time Deployment
### Prerequisites
- Docker + Docker Compose installed
- RAM ≥ 8 GB
- Disk ≥ 50 GB
- Port 80, 443 accessible
### Step 1: Setup Network
```bash
# สร้าง Docker network ร่วม
docker network create shared_data_network
```
### Step 2: Configure Environment
```bash
# Copy และแก้ไขค่า
cp .env.example .env.global
nano .env.global
# Supabase env
cd 02-supabase
cp .env.example .env
nano .env
# API Service env
cd ../03-apiservice
cp .env.example .env
nano .env
# MinIO env
cd ../07-minio
cp .env.example .env
nano .env
```
### Step 3: Start Services (ตามลำดับ)
```bash
# 1. Infrastructure (Nginx + Keycloak + PostgreSQL + Redis)
cd 01-infra
docker compose --env-file ../.env.global up -d
# รอ PostgreSQL พร้อม (~30 วินาที)
sleep 30
# 2. Supabase
cd ../02-supabase
docker compose up -d
# รอ Supabase DB พร้อม (~60 วินาที)
sleep 60
# 3. API Service
cd ../03-apiservice
docker compose --env-file ../.env.global up --build -d
# 4. Airflow (ถ้าใช้งาน)
cd ../05-airflow
docker compose up airflow-init # รอให้ init เสร็จ
docker compose up -d
# 5. Analytics (Superset)
cd ../06-analytics
docker compose --env-file ../.env.global build
docker compose --env-file ../.env.global up -d
# 6. MinIO
cd ../07-minio
docker compose up -d
```
### Step 4: Verify Services
```bash
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"
```
Expected containers:
- `nginx-proxy-manager`
- `keycloak`
- `postgres`
- `redis`
- `dozzle`
- `sdp-supabase-db`
- `sdp-supabase-studio`
- `sdp-supabase-kong`
- `sdp-supabase-auth`
- `sdp-supabase-rest`
- `sdp-realtime-dev`
- `sdp-supabase-storage`
- `sdp-supabase-pooler`
- `sdp-supabase-analytics`
- `sdp-supabase-meta`
- `apiservice`
- `superset`
- `minio`
---
## Post-Installation Setup
### Setup Keycloak
1. เข้า `https://ai.sriphat.com/keycloak/admin`
2. Login ด้วย `KEYCLOAK_ADMIN` credentials
3. สร้าง Realm: `sriphat`
4. สร้าง Clients:
- `apiservice` (API Service)
- `superset-client` (Apache Superset)
- `minio-client` (MinIO)
- `airflow-client` (Apache Airflow)
5. เชื่อมต่อ LDAP/AD (optional)
### Initialize API Service
```bash
# เข้า Admin UI
https://ai.sriphat.com/apiservice/admin/
# สร้าง API Client และ Generate Key
curl -X POST "https://ai.sriphat.com/apiservice/admin/api-keys/generate" \
-H "Cookie: session=<admin-session>" \
-d "client_id=1&permissions=feed.checkpoint:write&name=his-key"
```
### Setup Superset Data Sources
1. เข้า `https://ai.sriphat.com/superset`
2. Settings → Database Connections → Add
3. เพิ่ม PostgreSQL:
```
postgresql://postgres:<password>@postgres:5432/postgres
```
---
## Daily Operations
### Start All Services
```bash
# Infrastructure
cd 01-infra && docker compose --env-file ../.env.global up -d && cd ..
# Supabase
cd 02-supabase && docker compose up -d && cd ..
# API Service
cd 03-apiservice && docker compose --env-file ../.env.global up -d && cd ..
# Airflow
cd 05-airflow && docker compose up -d && cd ..
# Analytics
cd 06-analytics && docker compose --env-file ../.env.global up -d && cd ..
# MinIO
cd 07-minio && docker compose up -d && cd ..
```
### Stop All Services
```bash
cd 01-infra && docker compose down && cd ..
cd 02-supabase && docker compose down && cd ..
cd 03-apiservice && docker compose down && cd ..
cd 05-airflow && docker compose down && cd ..
cd 06-analytics && docker compose down && cd ..
cd 07-minio && docker compose down && cd ..
```
---
## Backup & Restore
### Backup PostgreSQL (Infra)
```bash
# Backup ทั้ง database
docker exec postgres pg_dumpall -U postgres > backup_all_$(date +%Y%m%d_%H%M).sql
# Backup เฉพาะ database
docker exec postgres pg_dump -U postgres postgres > backup_postgres_$(date +%Y%m%d).sql
docker exec postgres pg_dump -U postgres superset > backup_superset_$(date +%Y%m%d).sql
docker exec postgres pg_dump -U postgres airflow > backup_airflow_$(date +%Y%m%d).sql
```
### Backup Supabase PostgreSQL
```bash
docker exec sdp-supabase-db pg_dump -U postgres postgres > backup_supabase_$(date +%Y%m%d).sql
```
### Backup MinIO
```bash
# ใช้ mc mirror
mc mirror sriphat/ ./minio-backup-$(date +%Y%m%d)/
# หรือ tar data directory
tar -czf minio-backup-$(date +%Y%m%d).tar.gz 07-minio/data/
```
### Restore PostgreSQL
```bash
# Restore
docker exec -i postgres psql -U postgres postgres < backup_postgres_20260501.sql
```
---
## Update Services
```bash
# Pull latest images
cd 01-infra && docker compose --env-file ../.env.global pull && cd ..
cd 02-supabase && docker compose pull && cd ..
cd 06-analytics && docker compose --env-file ../.env.global pull && cd ..
cd 07-minio && docker compose pull && cd ..
# Rebuild API Service (มี code changes)
cd 03-apiservice
docker compose --env-file ../.env.global build
docker compose --env-file ../.env.global up -d
```
---
## Troubleshooting
### PostgreSQL ไม่ start / connection refused
```bash
# ตรวจสอบ health
docker exec postgres pg_isready -U postgres
# ดู logs
docker logs postgres --tail 50
# Check schemas
docker exec postgres psql -U postgres -c "\dn"
```
### Keycloak ไม่ start
```bash
# ดู logs (มักเกิดจาก PostgreSQL ยังไม่พร้อม)
docker logs keycloak --tail 50
# Restart หลัง PostgreSQL พร้อม
docker restart keycloak
```
### API Service ไม่ connect database
```bash
# ตรวจสอบ network
docker network inspect shared_data_network
# ตรวจสอบ env vars
docker exec apiservice env | grep DB_
# Test connection จาก container
docker exec apiservice python -c "import psycopg2; psycopg2.connect(host='postgres', user='postgres', password='<pass>')"
```
### Supabase services unhealthy
```bash
# ตรวจสอบทุก container
docker ps --filter "name=sdp-"
# Restart ตามลำดับ dependency
docker restart sdp-supabase-db
sleep 10
docker restart sdp-supabase-analytics
sleep 10
docker restart sdp-supabase-kong
docker restart sdp-supabase-auth
docker restart sdp-supabase-rest
```
### Airflow worker ไม่ pick tasks
```bash
# ตรวจสอบ Redis connectivity
docker exec airflow-worker redis-cli -h redis ping
# ตรวจสอบ worker
docker logs airflow-worker --tail 50
# Restart worker
docker restart airflow-worker
```
### Nginx 502 Bad Gateway
```bash
# ตรวจสอบว่า backend container ทำงานอยู่
docker ps | grep <service-name>
# ตรวจสอบ logs
docker logs nginx-proxy-manager --tail 50
docker logs <service-name> --tail 50
# ตรวจสอบ network
docker network inspect shared_data_network | grep -A5 <service-name>
```
### MinIO ไม่ accessible
```bash
# Health check
curl -f http://localhost:9000/minio/health/live
# Logs
docker logs minio --tail 50
# Disk space
df -h
```
---
## Monitoring
### Dozzle (Docker Logs)
**URL:** `https://ai.sriphat.com/dozzle`
Dozzle monitor ทั้ง:
- Main server (local)
- Remote agent: `192.168.100.9:7007`
### Container Health
```bash
# ดู health status
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Health}}"
# ดู resource usage
docker stats --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}"
```
---
## Access Points Summary
| Service | URL | Credentials |
|---------|-----|-------------|
| Nginx Proxy | `http://192.168.100.9:8020` | — |
| Keycloak Admin | `https://ai.sriphat.com/keycloak/admin` | `KEYCLOAK_ADMIN` |
| Supabase Studio | `https://ai.sriphat.com/supabase` | DB credentials |
| Supabase API | `https://ai.sriphat.com/supabase-api` | ANON_KEY / SERVICE_ROLE_KEY |
| API Service | `https://ai.sriphat.com/apiservice` | Admin credentials |
| Airflow | `https://ai.sriphat.com/airflow` | Airflow admin |
| Superset | `https://ai.sriphat.com/superset` | `SUPERSET_ADMIN_*` |
| MinIO Console | `https://ai.sriphat.com/minio-console` | `MINIO_ROOT_*` |
| MinIO API | `https://ai.sriphat.com/minio` | S3 credentials |
| Dozzle | `https://ai.sriphat.com/dozzle` | — (no auth default) |
---
## Related
- [[00-Project-Overview]]
- [[01-Infrastructure]]
- [[07-Security-Strategy]]

View File

@@ -0,0 +1,111 @@
---
tags:
- project/sriphat
- reference
- ports
- network
created: 2026-05-07
status: active
---
# Port & Service Reference
## Host Ports (External Access)
| Port | Service | Container | Protocol |
|------|---------|-----------|---------|
| **8020** | Nginx Reverse Proxy | `nginx-proxy-manager` | HTTP |
| **8085** | Keycloak | `keycloak` | HTTP |
| **5435** | PostgreSQL (Infra) | `postgres` | TCP |
| **9999** | Dozzle (Log Monitor) | `dozzle` | HTTP |
| **3010** | Supabase Studio | `sdp-supabase-studio` | HTTP |
| **8100** | Supabase Kong API | `sdp-supabase-kong` | HTTP |
| **8444** | Supabase Kong API | `sdp-supabase-kong` | HTTPS |
| **5434** | Supabase PostgreSQL | `sdp-supabase-db` | TCP |
| **6544** | Supabase Pooler | `sdp-supabase-pooler` | TCP |
| **8040** | API Service | `apiservice` | HTTP |
| **8200** | Airflow API Server | `airflow-apiserver` | HTTP |
| **5555** | Flower (Celery UI) | `flower` | HTTP (optional) |
| **8088** | Apache Superset | `superset` | HTTP |
| **9000** | MinIO API | `minio` | HTTP |
| **9001** | MinIO Console | `minio` | HTTP |
## Internal-Only Ports (Docker Network)
| Port | Service | Container | ใช้งาน |
|------|---------|-----------|--------|
| **5432** | PostgreSQL (Infra) | `postgres` | Keycloak, API Service, Superset, Airflow |
| **6379** | Redis | `redis` | Airflow Celery broker |
| **8080** | Keycloak | `keycloak` | Internal (Nginx proxy → external 8085) |
| **9999** | GoTrue (Auth) | `sdp-supabase-auth` | Supabase auth |
| **3000** | PostgREST | `sdp-supabase-rest` | Supabase REST API |
| **4000** | Realtime | `sdp-realtime-dev` | WebSocket |
| **5000** | Storage API | `sdp-supabase-storage` | File storage |
| **5001** | ImgProxy | `sdp-supabase-imgproxy` | Image transform |
| **8080** | Postgres Meta | `sdp-supabase-meta` | DB metadata API |
| **4000** | Logflare | `sdp-supabase-analytics` | Log analytics |
| **4000** | Supavisor | `sdp-supabase-pooler` | Pooler management |
| **9001** | Vector | `sdp-supabase-vector` | Health check |
| **8080** | Airflow Scheduler | `airflow-scheduler` | Health check |
| **8974** | Airflow Scheduler | `airflow-scheduler` | Health check endpoint |
## Nginx Subpath Routing
| Subpath | Backend Container | Port |
|---------|-----------------|------|
| `/apiservice` | `apiservice` | 8040 |
| `/keycloak` | `keycloak` | 8080 |
| `/supabase` | `sdp-supabase-studio` | 3000 |
| `/supabase-api` | `sdp-supabase-kong` | 8000 |
| `/superset` | `superset` | 8088 |
| `/airflow` | `airflow-apiserver` | 8080 |
| `/minio` | `minio` | 9000 |
| `/minio-console` | `minio` | 9001 |
| `/dozzle` | `dozzle` | 8080 |
## DNS / Hosts
| Name | IP | ใช้งาน |
|------|----|--------|
| `dev.sriphat.com` | `192.168.100.9` | extra_hosts ใน containers |
| `ai.sriphat.com` | ตาม production DNS | Production URL |
## Docker Network
```
Network: shared_data_network (external, bridge)
Containers ที่ join:
├── nginx-proxy-manager
├── keycloak
├── postgres
├── redis
├── dozzle
├── sdp-supabase-studio
├── sdp-supabase-kong
├── sdp-supabase-auth
├── sdp-supabase-rest
├── sdp-realtime-dev
├── sdp-supabase-storage
├── sdp-supabase-imgproxy
├── sdp-supabase-meta
├── sdp-supabase-edge-functions
├── sdp-supabase-analytics
├── sdp-supabase-db
├── sdp-supabase-vector
├── sdp-supabase-pooler
├── apiservice
├── airflow-apiserver
├── airflow-scheduler
├── airflow-dag-processor
├── airflow-worker
├── airflow-triggerer
├── superset
└── minio
```
## Related
- [[00-Project-Overview]]
- [[01-Infrastructure]]
- [[08-Operations-Runbook]]