update configuration docker setup for data platform
This commit is contained in:
jigoong
@@ -32,3 +32,18 @@ ADMIN_PASSWORD=your-admin-password
|
||||
|
||||
# API Key Encryption (for storing encrypted keys in DB)
|
||||
API_KEY_ENC_SECRET=your-encryption-secret-key-here
|
||||
|
||||
# Debug settings (set to true for detailed logging, false for production)
|
||||
DEBUG_AUTH=false
|
||||
|
||||
# Keycloak Authentication (for web pages)
|
||||
KEYCLOAK_SERVER_URL=http://keycloak:8080
|
||||
KEYCLOAK_REALM=master
|
||||
KEYCLOAK_CLIENT_ID=apiservice
|
||||
KEYCLOAK_CLIENT_SECRET=your-keycloak-client-secret
|
||||
KEYCLOAK_REDIRECT_URI=http://localhost:8040/apiservice/auth/callback
|
||||
|
||||
# Airflow Integration
|
||||
AIRFLOW_API_URL=http://airflow-webserver:8080
|
||||
AIRFLOW_API_TOKEN=your-airflow-api-token
|
||||
AIRFLOW_DAG_ID_FINANCE=process_finance_excel
|
||||
|
||||
2
03-apiservice/.gitignore
vendored
2
03-apiservice/.gitignore
vendored
@@ -8,3 +8,5 @@ venv/
|
||||
.mypy_cache/
|
||||
ruff_cache/
|
||||
.windsurf/
|
||||
data/uploads/*
|
||||
!data/uploads/.gitkeep
|
||||
|
||||
334
03-apiservice/AIRFLOW_INTEGRATION.md
Normal file
334
03-apiservice/AIRFLOW_INTEGRATION.md
Normal file
@@ -0,0 +1,334 @@
|
||||
# Airflow Integration Guide
|
||||
|
||||
คู่มือการตั้งค่าและใช้งาน Airflow Integration สำหรับ Finance Excel Upload
|
||||
|
||||
## 📋 Overview
|
||||
|
||||
เมื่อ upload ไฟล์ Excel ผ่านหน้า `/data-management/finance` ระบบจะ:
|
||||
1. บันทึกไฟล์ลง `/data/uploads/`
|
||||
2. สร้าง record ใน database (`fastapi.upload_history`)
|
||||
3. **Trigger Airflow DAG** `process_finance_excel` (พร้อม retry 3 ครั้ง, รอ 10 วิ)
|
||||
4. Airflow จะ process ไฟล์และอัพเดท status กลับมา
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Configuration
|
||||
|
||||
### 1. สร้าง Airflow API Token
|
||||
|
||||
```bash
|
||||
# เข้า Airflow container
|
||||
docker exec -it airflow-webserver bash
|
||||
|
||||
# สร้าง API token
|
||||
airflow users create \
|
||||
--username apiservice \
|
||||
--firstname API \
|
||||
--lastname Service \
|
||||
--role Admin \
|
||||
--email apiservice@sriphat.com \
|
||||
--password your-secure-password
|
||||
|
||||
# หรือใช้ CLI สร้าง token
|
||||
python -c "from airflow.api.auth.backend.basic_auth import BasicAuth; print(BasicAuth().get_token('apiservice', 'your-secure-password'))"
|
||||
```
|
||||
|
||||
**หรือใช้ Airflow UI:**
|
||||
1. เข้า `http://ai.sriphat.com/airflow`
|
||||
2. ไปที่ **Security** → **List Users**
|
||||
3. เลือก user → **Edit**
|
||||
4. Copy **API Token** (หรือสร้างใหม่)
|
||||
|
||||
### 2. อัพเดท Environment Variables
|
||||
|
||||
แก้ไข `.env` ใน `03-apiservice/`:
|
||||
|
||||
```bash
|
||||
# Airflow Integration
|
||||
AIRFLOW_API_URL=http://airflow-webserver:8080
|
||||
AIRFLOW_API_TOKEN=your-airflow-api-token-here
|
||||
AIRFLOW_DAG_ID_FINANCE=process_finance_excel
|
||||
```
|
||||
|
||||
### 3. Restart API Service
|
||||
|
||||
```bash
|
||||
cd 03-apiservice
|
||||
docker compose restart
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Database Schema Updates
|
||||
|
||||
### New Fields in `fastapi.upload_history`:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `airflow_dag_run_id` | String | DAG run ID จาก Airflow |
|
||||
| `airflow_state` | String | State: queued, running, success, failed |
|
||||
| `processing_started_at` | DateTime | เวลาเริ่ม process |
|
||||
| `processing_completed_at` | DateTime | เวลาเสร็จ |
|
||||
| `error_message` | Text | Error message (ถ้ามี) |
|
||||
|
||||
### Migration
|
||||
|
||||
```bash
|
||||
# Restart apiservice เพื่อสร้าง columns ใหม่
|
||||
cd 03-apiservice
|
||||
docker compose down
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 How It Works
|
||||
|
||||
### Upload Flow:
|
||||
|
||||
```
|
||||
1. User uploads Excel file
|
||||
↓
|
||||
2. API saves file to /data/uploads/
|
||||
↓
|
||||
3. Create UploadHistory record (status: pending)
|
||||
↓
|
||||
4. Trigger Airflow DAG (with retry logic)
|
||||
├─ Attempt 1: Try trigger
|
||||
├─ Wait 10 seconds
|
||||
├─ Attempt 2: Retry
|
||||
├─ Wait 10 seconds
|
||||
└─ Attempt 3: Final retry
|
||||
↓
|
||||
5. Update record:
|
||||
- Success: status=processing, airflow_dag_run_id=xxx
|
||||
- Failure: status=error, error_message=xxx
|
||||
↓
|
||||
6. Return response to frontend
|
||||
```
|
||||
|
||||
### Airflow DAG Flow:
|
||||
|
||||
```
|
||||
validate_file
|
||||
↓
|
||||
process_data
|
||||
↓
|
||||
load_to_database
|
||||
↓
|
||||
update_status
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 API Endpoints
|
||||
|
||||
### Upload File (Triggers Airflow)
|
||||
|
||||
```http
|
||||
POST /data-management/finance/upload
|
||||
Content-Type: multipart/form-data
|
||||
|
||||
file: [Excel file]
|
||||
description: "Optional description"
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"message": "File 'finance.xlsx' uploaded successfully",
|
||||
"upload_id": "upload_20260313_173000",
|
||||
"filename": "20260313_173000_finance.xlsx",
|
||||
"airflow_triggered": true,
|
||||
"dag_run_id": "manual__2026-03-13T10:30:00+00:00",
|
||||
"error": null
|
||||
}
|
||||
```
|
||||
|
||||
### Get Upload History
|
||||
|
||||
```http
|
||||
GET /data-management/finance/uploads
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
[
|
||||
{
|
||||
"id": "upload_20260313_173000",
|
||||
"filename": "finance.xlsx",
|
||||
"filepath": "/data/uploads/20260313_173000_finance.xlsx",
|
||||
"uploaded_at": "2026-03-13T17:30:00",
|
||||
"description": "Q1 2026 Finance Data",
|
||||
"status": "processing",
|
||||
"uploaded_by": "admin",
|
||||
"airflow_dag_run_id": "manual__2026-03-13T10:30:00+00:00",
|
||||
"airflow_state": "running",
|
||||
"processing_started_at": "2026-03-13T17:30:05",
|
||||
"processing_completed_at": null,
|
||||
"error_message": null
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Monitoring
|
||||
|
||||
### 1. Check Upload Status
|
||||
|
||||
```bash
|
||||
# ดู logs ของ API Service
|
||||
docker logs apiservice -f | grep "Airflow"
|
||||
|
||||
# ตัวอย่าง logs:
|
||||
# Triggering Airflow DAG (attempt 1/3)
|
||||
# Airflow DAG triggered successfully: manual__2026-03-13T10:30:00+00:00
|
||||
```
|
||||
|
||||
### 2. Check Airflow DAG
|
||||
|
||||
1. เข้า Airflow UI: `http://ai.sriphat.com/airflow`
|
||||
2. ไปที่ **DAGs** → `process_finance_excel`
|
||||
3. ดู **DAG Runs** และ **Task Instances**
|
||||
|
||||
### 3. Check Database
|
||||
|
||||
```sql
|
||||
-- ดู upload history พร้อม Airflow status
|
||||
SELECT
|
||||
upload_id,
|
||||
filename,
|
||||
status,
|
||||
airflow_dag_run_id,
|
||||
airflow_state,
|
||||
processing_started_at,
|
||||
processing_completed_at,
|
||||
error_message
|
||||
FROM fastapi.upload_history
|
||||
ORDER BY uploaded_at DESC
|
||||
LIMIT 10;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🐛 Troubleshooting
|
||||
|
||||
### Issue: "Failed to trigger Airflow after 3 attempts"
|
||||
|
||||
**สาเหตุ:**
|
||||
- Airflow API Token ไม่ถูกต้อง
|
||||
- Airflow webserver ไม่ทำงาน
|
||||
- Network connection ล้มเหลว
|
||||
|
||||
**วิธีแก้:**
|
||||
|
||||
```bash
|
||||
# 1. ตรวจสอบ Airflow webserver
|
||||
docker ps | grep airflow-webserver
|
||||
|
||||
# 2. ตรวจสอบ network
|
||||
docker network inspect shared_data_network
|
||||
|
||||
# 3. ทดสอบ API token
|
||||
curl -X GET \
|
||||
http://airflow-webserver:8080/api/v1/dags \
|
||||
-H "Authorization: Bearer YOUR_TOKEN"
|
||||
|
||||
# 4. ดู logs
|
||||
docker logs airflow-webserver -f
|
||||
docker logs apiservice -f
|
||||
```
|
||||
|
||||
### Issue: DAG ไม่ปรากฏใน Airflow UI
|
||||
|
||||
**สาเหตุ:**
|
||||
- DAG file มี syntax error
|
||||
- DAG ถูก pause
|
||||
- Airflow scheduler ไม่ทำงาน
|
||||
|
||||
**วิธีแก้:**
|
||||
|
||||
```bash
|
||||
# 1. ตรวจสอบ DAG file
|
||||
docker exec airflow-webserver airflow dags list | grep process_finance_excel
|
||||
|
||||
# 2. Test DAG
|
||||
docker exec airflow-webserver airflow dags test process_finance_excel
|
||||
|
||||
# 3. Unpause DAG
|
||||
docker exec airflow-webserver airflow dags unpause process_finance_excel
|
||||
|
||||
# 4. ตรวจสอบ scheduler
|
||||
docker logs airflow-scheduler -f
|
||||
```
|
||||
|
||||
### Issue: DAG triggered แต่ไม่ทำงาน
|
||||
|
||||
**สาเหตุ:**
|
||||
- Volume mount ไม่ถูกต้อง (Airflow เข้าถึง `/data/uploads` ไม่ได้)
|
||||
- Dependencies ขาดหาย (pandas, openpyxl)
|
||||
|
||||
**วิธีแก้:**
|
||||
|
||||
```bash
|
||||
# 1. ตรวจสอบ volume
|
||||
docker exec airflow-worker ls -la /data/uploads
|
||||
|
||||
# 2. ติดตั้ง dependencies
|
||||
docker exec airflow-worker pip install pandas openpyxl
|
||||
|
||||
# หรือเพิ่มใน docker-compose.yaml:
|
||||
# _PIP_ADDITIONAL_REQUIREMENTS: "pandas openpyxl"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔐 Security Notes
|
||||
|
||||
1. **API Token Security:**
|
||||
- เก็บ token ใน `.env` (ไม่ commit ลง git)
|
||||
- ใช้ token ที่มี scope จำกัด (ไม่ใช่ admin token)
|
||||
- Rotate token เป็นระยะ
|
||||
|
||||
2. **File Access:**
|
||||
- Airflow ต้องมี read access ไปยัง `/data/uploads`
|
||||
- ตรวจสอบ file permissions
|
||||
|
||||
3. **Network:**
|
||||
- API Service และ Airflow ต้องอยู่ใน `shared_data_network`
|
||||
- ใช้ internal network (ไม่ expose Airflow API ออกภายนอก)
|
||||
|
||||
---
|
||||
|
||||
## 📚 References
|
||||
|
||||
- [Airflow REST API Documentation](https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html)
|
||||
- [Airflow Authentication](https://airflow.apache.org/docs/apache-airflow/stable/security/api.html)
|
||||
- [FastAPI Background Tasks](https://fastapi.tiangolo.com/tutorial/background-tasks/)
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Next Steps
|
||||
|
||||
1. **Implement DAG Logic:**
|
||||
- แก้ไข `05-airflow/dags/process_finance_excel.py`
|
||||
- เพิ่ม data transformation logic
|
||||
- เพิ่ม database loading logic
|
||||
|
||||
2. **Add Callback:**
|
||||
- สร้าง endpoint `/api/v1/airflow/callback/{upload_id}`
|
||||
- Airflow จะ callback เมื่อเสร็จ/ล้มเหลว
|
||||
- อัพเดท `processing_completed_at` และ `airflow_state`
|
||||
|
||||
3. **Frontend Updates:**
|
||||
- เพิ่ม real-time status polling
|
||||
- แสดง Airflow state badges
|
||||
- Link ไปยัง Airflow DAG run
|
||||
|
||||
4. **Testing:**
|
||||
- ทดสอบ upload file
|
||||
- ตรวจสอบ Airflow trigger
|
||||
- ตรวจสอบ retry logic
|
||||
- ทดสอบ error handling
|
||||
325
03-apiservice/KEYCLOAK-SETUP.md
Normal file
325
03-apiservice/KEYCLOAK-SETUP.md
Normal file
@@ -0,0 +1,325 @@
|
||||
# Keycloak Setup Guide for API Service Authentication
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Keycloak must be running at `http://localhost:8085`
|
||||
2. PostgreSQL database must have `keycloak` database created
|
||||
3. API Service must have Keycloak configuration in `.env`
|
||||
|
||||
## Step 1: Restart Infrastructure to Create Keycloak Database
|
||||
|
||||
```bash
|
||||
cd 01-infra
|
||||
docker compose down
|
||||
docker compose --env-file ../.env.global up -d
|
||||
```
|
||||
|
||||
Wait for Keycloak to initialize (check logs):
|
||||
```bash
|
||||
docker logs keycloak -f
|
||||
```
|
||||
|
||||
Look for: `Keycloak ... started`
|
||||
|
||||
## Step 2: Access Keycloak Admin Console
|
||||
|
||||
1. Open browser: `http://localhost:8085/admin`
|
||||
2. Login credentials:
|
||||
- **Username**: `admin`
|
||||
- **Password**: `admin_secret_pass_2026`
|
||||
|
||||
## Step 3: Create Client for API Service
|
||||
|
||||
### 3.1 Navigate to Clients
|
||||
1. In left sidebar, click **Clients**
|
||||
2. Click **Create client** button
|
||||
|
||||
### 3.2 General Settings
|
||||
- **Client type**: `OpenID Connect`
|
||||
- **Client ID**: `apiservice`
|
||||
- Click **Next**
|
||||
|
||||
### 3.3 Capability Config
|
||||
- **Client authentication**: `ON` (toggle to enable)
|
||||
- **Authorization**: `OFF`
|
||||
- **Authentication flow**:
|
||||
- ✅ Standard flow
|
||||
- ✅ Direct access grants
|
||||
- ❌ Implicit flow (uncheck)
|
||||
- ❌ Service accounts roles (uncheck)
|
||||
- Click **Next**
|
||||
|
||||
### 3.4 Login Settings
|
||||
Fill in the following URLs:
|
||||
|
||||
**Root URL**:
|
||||
```
|
||||
http://localhost:8040
|
||||
```
|
||||
|
||||
**Home URL**:
|
||||
```
|
||||
http://localhost:8040/apiservice/
|
||||
```
|
||||
|
||||
**Valid redirect URIs** (add both):
|
||||
```
|
||||
http://localhost:8040/apiservice/auth/callback
|
||||
https://ai.sriphat.com/apiservice/auth/callback
|
||||
```
|
||||
|
||||
**Valid post logout redirect URIs** (add both):
|
||||
```
|
||||
http://localhost:8040/apiservice/
|
||||
https://ai.sriphat.com/apiservice/
|
||||
```
|
||||
|
||||
**Web origins** (add both):
|
||||
```
|
||||
http://localhost:8040
|
||||
https://ai.sriphat.com
|
||||
```
|
||||
|
||||
Click **Save**
|
||||
|
||||
### 3.5 Get Client Secret
|
||||
1. Go to **Credentials** tab
|
||||
2. Copy the **Client secret** value
|
||||
3. Save it - you'll need it for the `.env` file
|
||||
|
||||
## Step 4: Configure API Service
|
||||
|
||||
### 4.1 Create/Update `.env` file in `03-apiservice`
|
||||
|
||||
```bash
|
||||
cd ../03-apiservice
|
||||
```
|
||||
|
||||
Create `.env` file with:
|
||||
|
||||
```bash
|
||||
# Application
|
||||
APP_NAME=APIsService
|
||||
ROOT_PATH=/apiservice
|
||||
|
||||
# Timezone
|
||||
TIMEZONE=Asia/Bangkok
|
||||
|
||||
# PostgreSQL Database
|
||||
DB_HOST=postgres
|
||||
DB_PORT=5432
|
||||
DB_USER=postgres
|
||||
DB_PASSWORD=Secure_Hospital_Pass_2026
|
||||
DB_NAME=postgres
|
||||
DB_SSLMODE=prefer
|
||||
|
||||
# Supabase (if not using, keep dummy values)
|
||||
SUPABASE_DB_HOST=localhost
|
||||
SUPABASE_DB_PORT=5432
|
||||
SUPABASE_DB_USER=postgres
|
||||
SUPABASE_DB_PASSWORD=dummy
|
||||
SUPABASE_DB_NAME=postgres
|
||||
SUPABASE_DB_SSLMODE=disable
|
||||
SUPABASE_API_URL=http://localhost:8000
|
||||
SUPABASE_API_KEY=dummy
|
||||
|
||||
# Admin Authentication
|
||||
ADMIN_SECRET_KEY=apiservice_admin_secret_2026
|
||||
ADMIN_USERNAME=admin
|
||||
ADMIN_PASSWORD=change_me_2026
|
||||
|
||||
# API Key Encryption
|
||||
API_KEY_ENC_SECRET=dev_encryption_secret_2026
|
||||
|
||||
# Keycloak Authentication
|
||||
KEYCLOAK_SERVER_URL=http://keycloak:8080
|
||||
KEYCLOAK_REALM=master
|
||||
KEYCLOAK_CLIENT_ID=apiservice
|
||||
KEYCLOAK_CLIENT_SECRET=<PASTE_YOUR_CLIENT_SECRET_HERE>
|
||||
KEYCLOAK_REDIRECT_URI=http://localhost:8040/apiservice/auth/callback
|
||||
```
|
||||
|
||||
**Important**: Replace `<PASTE_YOUR_CLIENT_SECRET_HERE>` with the actual secret from Step 3.5
|
||||
|
||||
## Step 5: Create Test User in Keycloak
|
||||
|
||||
### 5.1 Create User
|
||||
1. In Keycloak Admin, go to **Users**
|
||||
2. Click **Add user**
|
||||
3. Fill in:
|
||||
- **Username**: `testuser`
|
||||
- **Email**: `test@sriphat.local`
|
||||
- **First name**: `Test`
|
||||
- **Last name**: `User`
|
||||
- **Email verified**: `ON` (toggle)
|
||||
4. Click **Create**
|
||||
|
||||
### 5.2 Set Password
|
||||
1. Go to **Credentials** tab
|
||||
2. Click **Set password**
|
||||
3. Fill in:
|
||||
- **Password**: `password`
|
||||
- **Password confirmation**: `password`
|
||||
- **Temporary**: `OFF` (toggle off)
|
||||
4. Click **Save**
|
||||
5. Confirm by clicking **Save password**
|
||||
|
||||
## Step 6: Rebuild and Restart API Service
|
||||
|
||||
```bash
|
||||
cd 03-apiservice
|
||||
docker compose down
|
||||
docker compose --env-file ../.env up --build -d
|
||||
```
|
||||
|
||||
Check logs:
|
||||
```bash
|
||||
docker logs apiservice -f
|
||||
```
|
||||
|
||||
Look for successful startup without errors.
|
||||
|
||||
## Step 7: Test Authentication Flow
|
||||
|
||||
### 7.1 Access Landing Page
|
||||
1. Open browser: `http://localhost:8040/apiservice/`
|
||||
2. You should be redirected to Keycloak login page
|
||||
|
||||
### 7.2 Login
|
||||
1. Enter credentials:
|
||||
- **Username**: `testuser`
|
||||
- **Password**: `password`
|
||||
2. Click **Sign In**
|
||||
|
||||
### 7.3 Verify Success
|
||||
1. You should be redirected back to the landing page
|
||||
2. You should see your name in the top-right corner: "👤 Test User"
|
||||
3. You should see a **Logout** button
|
||||
|
||||
### 7.4 Test Protected Pages
|
||||
Try accessing:
|
||||
- `/apiservice/` - Landing page (requires auth)
|
||||
- `/apiservice/docs` - API docs (requires auth)
|
||||
- `/apiservice/data-management/finance` - Finance page (requires auth)
|
||||
|
||||
All should work without redirecting to login again.
|
||||
|
||||
### 7.5 Test API Endpoints (Should NOT require Keycloak auth)
|
||||
```bash
|
||||
# API endpoints use API Key authentication, not Keycloak
|
||||
curl -X GET http://localhost:8040/apiservice/api/v1/health
|
||||
```
|
||||
|
||||
This should work without Keycloak authentication.
|
||||
|
||||
### 7.6 Test Logout
|
||||
1. Click **Logout** button
|
||||
2. You should be logged out from Keycloak
|
||||
3. Accessing any protected page should redirect to login again
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Error: "access_denied" in Keycloak logs
|
||||
**Cause**: Client not configured correctly or client secret mismatch
|
||||
|
||||
**Solution**:
|
||||
1. Verify client ID is exactly `apiservice`
|
||||
2. Verify client secret in `.env` matches Keycloak
|
||||
3. Verify redirect URIs are correct
|
||||
4. Restart API service after changing `.env`
|
||||
|
||||
### Error: "SessionMiddleware must be installed"
|
||||
**Cause**: Middleware order is incorrect
|
||||
|
||||
**Solution**: Already fixed in `app/main.py` - SessionMiddleware is added after WebAuthenticationMiddleware
|
||||
|
||||
### Error: "Invalid redirect_uri"
|
||||
**Cause**: Redirect URI not in Keycloak's allowed list
|
||||
|
||||
**Solution**:
|
||||
1. Go to Keycloak → Clients → apiservice → Settings
|
||||
2. Add the exact redirect URI to "Valid redirect URIs"
|
||||
3. Make sure it matches `KEYCLOAK_REDIRECT_URI` in `.env`
|
||||
|
||||
### Error: Connection refused to Keycloak
|
||||
**Cause**: Keycloak not running or wrong URL
|
||||
|
||||
**Solution**:
|
||||
1. Check Keycloak is running: `docker ps | grep keycloak`
|
||||
2. Verify `KEYCLOAK_SERVER_URL=http://keycloak:8080` (use container name, not localhost)
|
||||
3. Verify both services are on same Docker network
|
||||
|
||||
## Production Deployment
|
||||
|
||||
For production deployment:
|
||||
|
||||
1. **Use HTTPS**:
|
||||
```bash
|
||||
KEYCLOAK_REDIRECT_URI=https://ai.sriphat.com/apiservice/auth/callback
|
||||
```
|
||||
|
||||
2. **Update Keycloak Client**:
|
||||
- Add production redirect URIs
|
||||
- Remove localhost URIs
|
||||
|
||||
3. **Secure Cookies**:
|
||||
- SessionMiddleware should use `secure=True, httponly=True, samesite='lax'`
|
||||
|
||||
4. **Use Nginx Proxy Manager**:
|
||||
- Configure SSL certificates
|
||||
- Set up proper reverse proxy rules
|
||||
|
||||
5. **Environment Variables**:
|
||||
- Use Docker secrets or external secret management
|
||||
- Never commit `.env` with real secrets to git
|
||||
|
||||
## Security Notes
|
||||
|
||||
1. **Client Secret**: Keep this secret! Never commit to git
|
||||
2. **Session Secret**: Use strong random value for `ADMIN_SECRET_KEY`
|
||||
3. **HTTPS**: Always use HTTPS in production
|
||||
4. **Token Expiration**: Configure appropriate token lifetimes in Keycloak
|
||||
5. **CORS**: Restrict `allow_origins` to known domains only
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
User Browser
|
||||
↓
|
||||
↓ (1) Access protected page
|
||||
↓
|
||||
API Service (FastAPI)
|
||||
↓
|
||||
↓ (2) Check session - not authenticated
|
||||
↓
|
||||
↓ (3) Redirect to Keycloak login
|
||||
↓
|
||||
Keycloak (OAuth2/OIDC)
|
||||
↓
|
||||
↓ (4) User enters credentials
|
||||
↓
|
||||
↓ (5) Redirect back with auth code
|
||||
↓
|
||||
API Service
|
||||
↓
|
||||
↓ (6) Exchange code for tokens
|
||||
↓
|
||||
↓ (7) Store user info in session
|
||||
↓
|
||||
↓ (8) Redirect to original page
|
||||
↓
|
||||
User sees protected page with user info
|
||||
```
|
||||
|
||||
## API vs Web Authentication
|
||||
|
||||
| Type | Authentication Method | Protected Routes |
|
||||
|------|----------------------|------------------|
|
||||
| **Web Pages** | Keycloak (OAuth2/OIDC) | `/`, `/docs`, `/data-management/*` |
|
||||
| **API Endpoints** | API Key (Bearer Token) | `/api/v1/*` |
|
||||
| **Admin Panel** | SQLAdmin (Basic Auth) | `/admin/*` |
|
||||
|
||||
This separation allows:
|
||||
- Human users to login via Keycloak for web UI
|
||||
- Applications/services to use API Keys for programmatic access
|
||||
- Admins to manage API keys via separate admin panel
|
||||
226
03-apiservice/README-PAGES.md
Normal file
226
03-apiservice/README-PAGES.md
Normal file
@@ -0,0 +1,226 @@
|
||||
# Web Pages Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
The API service now includes web pages for easy access to all platform services and data management functionality.
|
||||
|
||||
## Pages
|
||||
|
||||
### 1. Landing Page (`/`)
|
||||
|
||||
Main dashboard with navigation to all services:
|
||||
- **Supabase**: PostgreSQL database with REST API
|
||||
- **API Docs**: Interactive API documentation
|
||||
- **Airflow**: Workflow orchestration
|
||||
- **Airbyte**: Data integration and ETL
|
||||
- **Data Management**: File upload and processing (with submenu)
|
||||
- **DBT**: Data transformation
|
||||
- **Superset**: Business intelligence and visualization
|
||||
|
||||
### 2. Finance Excel Upload (`/data-management/finance`)
|
||||
|
||||
Upload and process financial Excel files with the following features:
|
||||
|
||||
**Upload Form:**
|
||||
- Drag-and-drop or click to select Excel files (.xlsx, .xls)
|
||||
- Optional description field
|
||||
- File validation and size display
|
||||
|
||||
**Upload Processing:**
|
||||
- Files saved to `/data/uploads/` with timestamp prefix
|
||||
- Unique filename generation to prevent conflicts
|
||||
- Automatic Airflow job triggering (when configured)
|
||||
|
||||
**Upload History:**
|
||||
- Real-time list of all uploaded files
|
||||
- Status tracking: pending, processing, success, error
|
||||
- Expandable accordion for logs and error details
|
||||
- Auto-refresh every 10 seconds
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Page Routes
|
||||
|
||||
```
|
||||
GET / - Landing page
|
||||
GET /data-management/finance - Finance upload page
|
||||
POST /data-management/finance/upload - Upload file endpoint
|
||||
GET /data-management/finance/uploads - Get all uploads
|
||||
GET /data-management/finance/uploads/{upload_id} - Get specific upload
|
||||
```
|
||||
|
||||
### Upload Endpoint
|
||||
|
||||
**POST** `/data-management/finance/upload`
|
||||
|
||||
**Form Data:**
|
||||
- `file`: Excel file (.xlsx or .xls)
|
||||
- `description`: Optional description text
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"message": "File 'example.xlsx' uploaded successfully",
|
||||
"upload_id": "upload_20260306_174500",
|
||||
"filename": "20260306_174500_example.xlsx"
|
||||
}
|
||||
```
|
||||
|
||||
**Error Response:**
|
||||
```json
|
||||
{
|
||||
"detail": "Invalid file type. Only .xlsx and .xls files are allowed."
|
||||
}
|
||||
```
|
||||
|
||||
## File Storage
|
||||
|
||||
Uploaded files are stored in:
|
||||
```
|
||||
/data/uploads/YYYYMMDD_HHMMSS_original_filename.xlsx
|
||||
```
|
||||
|
||||
Example:
|
||||
```
|
||||
/data/uploads/20260306_174530_finance_report.xlsx
|
||||
```
|
||||
|
||||
## Airflow Integration
|
||||
|
||||
The upload endpoint includes a placeholder for Airflow integration. To enable:
|
||||
|
||||
1. **Configure Airflow endpoint** in `app/routes/pages.py`:
|
||||
```python
|
||||
airflow_url = "http://airflow-webserver:8080/api/v1/dags/{dag_id}/dagRuns"
|
||||
```
|
||||
|
||||
2. **Set DAG ID** for the finance processing job
|
||||
|
||||
3. **Uncomment the Airflow trigger code** in `trigger_airflow_job()` function
|
||||
|
||||
4. **Configure authentication** (username/password or API token)
|
||||
|
||||
### Example Airflow Integration
|
||||
|
||||
```python
|
||||
async def trigger_airflow_job(filepath: str, upload_id: str) -> str:
|
||||
import httpx
|
||||
|
||||
airflow_url = "http://airflow-webserver:8080/api/v1/dags/finance_processing/dagRuns"
|
||||
headers = {"Content-Type": "application/json"}
|
||||
auth = ("airflow", "airflow")
|
||||
|
||||
payload = {
|
||||
"conf": {
|
||||
"filepath": filepath,
|
||||
"upload_id": upload_id
|
||||
}
|
||||
}
|
||||
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.post(
|
||||
airflow_url,
|
||||
json=payload,
|
||||
headers=headers,
|
||||
auth=auth
|
||||
)
|
||||
response.raise_for_status()
|
||||
result = response.json()
|
||||
return result["dag_run_id"]
|
||||
```
|
||||
|
||||
## Upload Status Tracking
|
||||
|
||||
Upload records include:
|
||||
- `id`: Unique identifier
|
||||
- `filename`: Original filename
|
||||
- `filepath`: Full path to saved file
|
||||
- `uploaded_at`: ISO timestamp
|
||||
- `description`: User-provided description
|
||||
- `status`: pending | processing | success | error
|
||||
- `job_id`: Airflow DAG run ID (when triggered)
|
||||
- `logs`: Error logs or processing information
|
||||
|
||||
## Database Storage
|
||||
|
||||
Currently uses in-memory storage (`uploads_db` list). For production:
|
||||
|
||||
1. **Create database table:**
|
||||
```sql
|
||||
CREATE TABLE uploads (
|
||||
id VARCHAR(100) PRIMARY KEY,
|
||||
filename VARCHAR(255) NOT NULL,
|
||||
filepath VARCHAR(500) NOT NULL,
|
||||
uploaded_at TIMESTAMP NOT NULL,
|
||||
description TEXT,
|
||||
status VARCHAR(20) NOT NULL,
|
||||
job_id VARCHAR(100),
|
||||
logs TEXT,
|
||||
created_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
```
|
||||
|
||||
2. **Replace in-memory storage** with SQLAlchemy model
|
||||
|
||||
3. **Add database queries** in route handlers
|
||||
|
||||
## Access URLs
|
||||
|
||||
- **Local**: http://localhost:8040/apiservice/
|
||||
- **Production**: https://ai.sriphat.com/apiservice/
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **File validation**: Only .xlsx and .xls files allowed
|
||||
2. **Filename sanitization**: Spaces replaced with underscores
|
||||
3. **Unique filenames**: Timestamp prefix prevents conflicts
|
||||
4. **File size limits**: Configure in nginx/FastAPI if needed
|
||||
5. **Authentication**: Add authentication middleware for production
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
1. **Database persistence**: Replace in-memory storage
|
||||
2. **File preview**: Show Excel data preview before processing
|
||||
3. **Batch upload**: Support multiple file uploads
|
||||
4. **Download processed files**: Allow downloading results
|
||||
5. **User management**: Track uploads by user
|
||||
6. **Email notifications**: Notify on processing completion
|
||||
7. **Scheduled jobs**: Automatic periodic processing
|
||||
8. **Data validation**: Validate Excel structure before processing
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Upload fails with 500 error
|
||||
- Check `/data/uploads/` directory exists and is writable
|
||||
- Verify volume mount in docker-compose.yml
|
||||
- Check container logs: `docker logs apiservice`
|
||||
|
||||
### Files not persisting after container restart
|
||||
- Ensure volume mount is configured: `./data/uploads:/data/uploads`
|
||||
- Check file permissions on host directory
|
||||
|
||||
### Airflow job not triggering
|
||||
- Verify Airflow is accessible from container
|
||||
- Check network connectivity: `docker network inspect shared_data_network`
|
||||
- Verify Airflow credentials and DAG ID
|
||||
- Check logs in upload history
|
||||
|
||||
## Development
|
||||
|
||||
To add new data management pages:
|
||||
|
||||
1. Create HTML template in `app/templates/`
|
||||
2. Add route in `app/routes/pages.py`
|
||||
3. Update landing page menu in `index.html`
|
||||
4. Add submenu item if needed
|
||||
|
||||
Example:
|
||||
```python
|
||||
@router.get("/data-management/hr", response_class=HTMLResponse)
|
||||
async def hr_page(request: Request):
|
||||
return templates.TemplateResponse(
|
||||
"data_management_hr.html",
|
||||
{"request": request, "root_path": settings.ROOT_PATH}
|
||||
)
|
||||
```
|
||||
@@ -9,9 +9,9 @@ from fastapi import APIRouter, Depends
|
||||
from sqlalchemy.dialects.postgresql import insert
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.api.v1.schemas import FeedCheckpointIn, FeedWaitingTimeIn
|
||||
from app.api.v1.schemas import FeedCheckpointIn, FeedWaitingTimeIn, PatientAppointmentIn
|
||||
from app.core.config import settings
|
||||
from app.db.models import RawOpdCheckpoint, RawWaitingTime
|
||||
from app.db.models import RawOpdCheckpoint, RawWaitingTime, PatientAppointment
|
||||
from app.security.dependencies import get_db, require_permission
|
||||
from app.utils.supabase_client import SupabaseAPIError, upsert_to_supabase_sync
|
||||
|
||||
@@ -21,6 +21,7 @@ router = APIRouter(prefix="/api/v1")
|
||||
|
||||
PERM_FEED_CHECKPOINT_WRITE = "feed.checkpoint:write"
|
||||
PERM_FEED_OLD_CHECKPOINT_WRITE = "feed.old-checkpoint:write"
|
||||
PERM_FEED_PATIENT_APPOINTMENT_WRITE = "feed.patient-appointment:write"
|
||||
|
||||
|
||||
def _to_tz(dt):
|
||||
@@ -220,3 +221,86 @@ def upsert_opd_checkpoint(
|
||||
"error": supabase_error,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@router.post("/feed/patient-appointment")
|
||||
def upsert_patient_appointment(
|
||||
payload: list[PatientAppointmentIn],
|
||||
_: Annotated[object, Depends(require_permission(PERM_FEED_PATIENT_APPOINTMENT_WRITE))],
|
||||
db: Annotated[Session, Depends(get_db)],
|
||||
):
|
||||
rows = []
|
||||
supabase_rows = []
|
||||
|
||||
for item in payload:
|
||||
# Prepare data for local database
|
||||
row = {
|
||||
"hn": item.hn,
|
||||
"txn": item.txn,
|
||||
"date": item.date, #+' 0:00:00'
|
||||
"time": _to_tz(datetime.strptime(item.date.strftime("%Y-%m-%d")+' '+item.time,'%Y-%m-%d %H:%M:%S')),
|
||||
"doctor_code": item.doctor_code,
|
||||
"period": item.period,
|
||||
"appointment_type": item.appointment_type,
|
||||
"updated_at": datetime.now(ZoneInfo(settings.TIMEZONE)),
|
||||
}
|
||||
rows.append(row)
|
||||
|
||||
# Prepare data for Supabase API (convert datetime to ISO string)
|
||||
supabase_row = {
|
||||
"hn": item.hn,
|
||||
"txn": item.txn,
|
||||
"date": _to_iso(_to_tz(datetime.strptime(item.date.strftime("%Y-%m-%d")+' 00:00:00+07:00','%Y-%m-%d %H:%M:%S%z'))),
|
||||
"time": _to_iso(_to_tz(datetime.strptime(item.date.strftime("%Y-%m-%d")+' '+item.time+'+07:00','%Y-%m-%d %H:%M:%S%z'))),
|
||||
"doctor_code": item.doctor_code,
|
||||
"period": item.period,
|
||||
"appointment_type": item.appointment_type,
|
||||
"updated_at": datetime.now(ZoneInfo(settings.TIMEZONE)).isoformat(),
|
||||
}
|
||||
supabase_rows.append(supabase_row)
|
||||
|
||||
# Insert/update to local database
|
||||
stmt = insert(PatientAppointment).values(rows)
|
||||
update_cols = {
|
||||
"txn": stmt.excluded.txn,
|
||||
"doctor_code": stmt.excluded.doctor_code,
|
||||
"period": stmt.excluded.period,
|
||||
"appointment_type": stmt.excluded.appointment_type,
|
||||
"updated_at": stmt.excluded.updated_at,
|
||||
}
|
||||
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=[PatientAppointment.hn, PatientAppointment.date, PatientAppointment.time],
|
||||
set_=update_cols,
|
||||
)
|
||||
result = db.execute(stmt)
|
||||
db.commit()
|
||||
|
||||
# Send data to Supabase via API call
|
||||
supabase_result = None
|
||||
supabase_error = None
|
||||
|
||||
try:
|
||||
logger.info(f"Sending {len(supabase_rows)} patient appointment records to Supabase API")
|
||||
supabase_result = upsert_to_supabase_sync(
|
||||
table="patient_appointment",
|
||||
data=supabase_rows,
|
||||
on_conflict="hn,date,time",
|
||||
)
|
||||
logger.info(f"Successfully sent patient appointment data to Supabase: {supabase_result.get('status_code')}")
|
||||
except SupabaseAPIError as e:
|
||||
logger.error(f"Failed to send patient appointment data to Supabase: {str(e)}")
|
||||
supabase_error = str(e)
|
||||
except Exception as e:
|
||||
logger.error(f"Unexpected error sending patient appointment data to Supabase: {str(e)}")
|
||||
supabase_error = f"Unexpected error: {str(e)}"
|
||||
|
||||
return {
|
||||
"upserted": len(rows),
|
||||
"rowcount": result.rowcount,
|
||||
"supabase": {
|
||||
"success": supabase_result is not None,
|
||||
"result": supabase_result,
|
||||
"error": supabase_error,
|
||||
},
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
from datetime import datetime
|
||||
from datetime import datetime, time, date
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
@@ -27,3 +27,13 @@ class FeedCheckpointIn(BaseModel):
|
||||
timestamp_out: datetime | None = None
|
||||
waiting_time: int | None = None
|
||||
bu: str | None = None
|
||||
|
||||
|
||||
class PatientAppointmentIn(BaseModel):
|
||||
hn: str
|
||||
txn: int | None = None
|
||||
date: date
|
||||
time: str
|
||||
doctor_code: str | None = None
|
||||
period: str | None = None
|
||||
appointment_type: str | None = None
|
||||
|
||||
@@ -33,5 +33,20 @@ class Settings(BaseSettings):
|
||||
|
||||
API_KEY_ENC_SECRET: str | None = None
|
||||
|
||||
# Debug settings
|
||||
DEBUG_AUTH: bool = False # Set to True to enable detailed authentication logging
|
||||
|
||||
# Keycloak Authentication (for web pages only)
|
||||
KEYCLOAK_SERVER_URL: str = "http://keycloak:8080"
|
||||
KEYCLOAK_REALM: str = "master"
|
||||
KEYCLOAK_CLIENT_ID: str = "apiservice"
|
||||
KEYCLOAK_CLIENT_SECRET: str = ""
|
||||
KEYCLOAK_REDIRECT_URI: str = "http://localhost:8040/apiservice/auth/callback"
|
||||
|
||||
# Airflow Integration
|
||||
AIRFLOW_API_URL: str = "http://airflow-webserver:8080"
|
||||
AIRFLOW_API_TOKEN: str = ""
|
||||
AIRFLOW_DAG_ID_FINANCE: str = "process_finance_excel"
|
||||
|
||||
|
||||
settings = Settings()
|
||||
|
||||
@@ -1,12 +1,54 @@
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.orm import Session
|
||||
import logging
|
||||
|
||||
from app.db.base import Base
|
||||
from app.db.engine import engine
|
||||
from app.models.user import User, Role # Import models to ensure they're registered
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def init_db() -> None:
|
||||
"""Initialize database schemas and tables"""
|
||||
with engine.begin() as conn:
|
||||
# Create schemas
|
||||
conn.execute(text("CREATE SCHEMA IF NOT EXISTS fastapi"))
|
||||
conn.execute(text("CREATE SCHEMA IF NOT EXISTS operationbi"))
|
||||
|
||||
conn.execute(text("CREATE SCHEMA IF NOT EXISTS rawdata"))
|
||||
|
||||
# Create all tables
|
||||
Base.metadata.create_all(bind=conn)
|
||||
logger.info("Database schemas and tables created")
|
||||
|
||||
# Seed default roles
|
||||
seed_roles()
|
||||
|
||||
|
||||
def seed_roles() -> None:
|
||||
"""Seed default roles if they don't exist"""
|
||||
from app.db.session import SessionLocal
|
||||
|
||||
db = SessionLocal()
|
||||
try:
|
||||
roles_data = [
|
||||
{"name": "admin", "description": "Full system access - can manage users and access all features"},
|
||||
{"name": "operation", "description": "Data management access - can upload and manage data"}
|
||||
]
|
||||
|
||||
for role_data in roles_data:
|
||||
existing = db.query(Role).filter(Role.name == role_data["name"]).first()
|
||||
if not existing:
|
||||
role = Role(**role_data)
|
||||
db.add(role)
|
||||
logger.info(f"Created role: {role_data['name']}")
|
||||
else:
|
||||
logger.info(f"Role already exists: {role_data['name']}")
|
||||
|
||||
db.commit()
|
||||
logger.info("Role seeding completed")
|
||||
except Exception as e:
|
||||
logger.error(f"Error seeding roles: {e}")
|
||||
db.rollback()
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@@ -46,6 +46,25 @@ class RawWaitingTime(Base):
|
||||
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
|
||||
|
||||
|
||||
class PatientAppointment(Base):
|
||||
__tablename__ = "patient_appointment"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("hn", "date", "time", name="uq_patient_appointment_hn_date_time"),
|
||||
{"schema": "rawdata"},
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True)
|
||||
hn: Mapped[str] = mapped_column(String(50), nullable=False)
|
||||
txn: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
||||
date: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
time: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
doctor_code: Mapped[str | None] = mapped_column(String(50), nullable=True)
|
||||
period: Mapped[str | None] = mapped_column(String(50), nullable=True)
|
||||
appointment_type: Mapped[str | None] = mapped_column(String(100), nullable=True)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
|
||||
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
|
||||
|
||||
|
||||
class ApiClient(Base):
|
||||
__tablename__ = "api_client"
|
||||
__table_args__ = {"schema": "fastapi"}
|
||||
|
||||
27
03-apiservice/app/db/session.py
Normal file
27
03-apiservice/app/db/session.py
Normal file
@@ -0,0 +1,27 @@
|
||||
"""
|
||||
Database session management
|
||||
"""
|
||||
from sqlalchemy.orm import sessionmaker, Session
|
||||
from app.db.engine import engine
|
||||
|
||||
# Create SessionLocal class
|
||||
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
|
||||
|
||||
|
||||
def get_db() -> Session:
|
||||
"""
|
||||
Dependency to get database session
|
||||
|
||||
Usage:
|
||||
@app.get("/items")
|
||||
def read_items(db: Session = Depends(get_db)):
|
||||
...
|
||||
|
||||
Yields:
|
||||
Database session
|
||||
"""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
yield db
|
||||
finally:
|
||||
db.close()
|
||||
@@ -12,6 +12,10 @@ import sqladmin
|
||||
|
||||
from app.admin import mount_admin
|
||||
from app.api.v1.routes import router as v1_router
|
||||
from app.routes.pages import router as pages_router
|
||||
from app.routes.auth import router as auth_router
|
||||
from app.routes.admin_users import router as admin_users_router
|
||||
from app.middleware.auth_middleware import WebAuthenticationMiddleware
|
||||
from app.core.config import settings
|
||||
from app.db.init_db import init_db
|
||||
|
||||
@@ -80,8 +84,9 @@ async def global_exception_handler(request, exc):
|
||||
status_code=500,
|
||||
content={"detail": "Internal server error", "error": str(exc)}
|
||||
)
|
||||
# Middleware order is important! They execute in reverse order (LIFO)
|
||||
# WebAuthenticationMiddleware needs SessionMiddleware, so SessionMiddleware must be added AFTER
|
||||
app.add_middleware(ForceHTTPSMiddleware)
|
||||
app.add_middleware(SessionMiddleware, secret_key=settings.ADMIN_SECRET_KEY)
|
||||
app.add_middleware(ForwardedProtoMiddleware)
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
@@ -90,7 +95,17 @@ app.add_middleware(
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
app.include_router(v1_router)
|
||||
# Add web authentication middleware (protects /, /docs, /data-management/* only)
|
||||
# API endpoints (/api/v1/*) continue to use API Key authentication
|
||||
app.add_middleware(WebAuthenticationMiddleware)
|
||||
# SessionMiddleware must be added AFTER middlewares that use it (due to LIFO execution)
|
||||
app.add_middleware(SessionMiddleware, secret_key=settings.ADMIN_SECRET_KEY)
|
||||
|
||||
app.include_router(v1_router) # API endpoints - use API Key auth
|
||||
app.include_router(pages_router) # Web pages - use Keycloak auth
|
||||
app.include_router(auth_router) # Authentication routes
|
||||
app.include_router(admin_users_router) # Admin user management API
|
||||
|
||||
app.mount("/admin/statics", StaticFiles(directory=statics_path), name="admin_statics")
|
||||
app.mount("/apiservice/admin/statics", StaticFiles(directory=statics_path), name="proxy_admin_statics")
|
||||
mount_admin(app)
|
||||
|
||||
1
03-apiservice/app/middleware/__init__.py
Normal file
1
03-apiservice/app/middleware/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
# Middleware package
|
||||
169
03-apiservice/app/middleware/auth_middleware.py
Normal file
169
03-apiservice/app/middleware/auth_middleware.py
Normal file
@@ -0,0 +1,169 @@
|
||||
"""
|
||||
Authentication middleware for web pages
|
||||
Protects web UI routes while allowing API endpoints to use API Key auth
|
||||
"""
|
||||
from fastapi import Request
|
||||
from fastapi.responses import RedirectResponse
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
from app.core.config import settings
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class WebAuthenticationMiddleware(BaseHTTPMiddleware):
|
||||
"""
|
||||
Middleware to enforce Keycloak authentication for web pages
|
||||
|
||||
Protected routes (require user login):
|
||||
- / (landing page)
|
||||
- /docs (API documentation)
|
||||
- /data-management/* (data management pages)
|
||||
|
||||
Excluded routes (no auth required or use different auth):
|
||||
- /auth/* (authentication endpoints)
|
||||
- /api/v1/* (API endpoints - use API Key auth)
|
||||
- /admin/* (SQLAdmin - has its own auth)
|
||||
- /apiservice/admin/statics/* (admin static files)
|
||||
- /admin/statics/* (admin static files)
|
||||
"""
|
||||
|
||||
# Routes that require user authentication
|
||||
PROTECTED_PATHS = [
|
||||
"/",
|
||||
"/docs",
|
||||
"/redoc",
|
||||
"/openapi.json",
|
||||
"/data-management"
|
||||
]
|
||||
|
||||
# Routes that are excluded from user authentication
|
||||
EXCLUDED_PATHS = [
|
||||
"/auth", # Authentication endpoints
|
||||
"/api/v1", # API endpoints (use API Key)
|
||||
"/admin", # SQLAdmin (has own auth)
|
||||
]
|
||||
|
||||
async def dispatch(self, request: Request, call_next):
|
||||
"""Process request and check authentication if needed"""
|
||||
|
||||
# Get the path without root_path prefix
|
||||
path = request.url.path
|
||||
|
||||
# Remove root_path prefix if it exists
|
||||
if settings.ROOT_PATH and path.startswith(settings.ROOT_PATH):
|
||||
path = path[len(settings.ROOT_PATH):]
|
||||
|
||||
# Ensure path starts with /
|
||||
if not path.startswith("/"):
|
||||
path = "/" + path
|
||||
|
||||
# Check if path is excluded from authentication
|
||||
is_excluded = any(path.startswith(excluded) for excluded in self.EXCLUDED_PATHS)
|
||||
|
||||
if is_excluded:
|
||||
# Skip authentication for excluded paths (including /api/v1/*)
|
||||
return await call_next(request)
|
||||
|
||||
# Check if path requires authentication
|
||||
requires_auth = any(
|
||||
path == protected or path.startswith(protected + "/")
|
||||
for protected in self.PROTECTED_PATHS
|
||||
)
|
||||
|
||||
if requires_auth:
|
||||
# Check if user is authenticated
|
||||
user = request.session.get("user")
|
||||
|
||||
if not user:
|
||||
# User not authenticated - redirect to login
|
||||
# Preserve the original path for redirect after login
|
||||
original_path = request.url.path
|
||||
|
||||
# Build login URL with redirect
|
||||
login_url = f"{settings.ROOT_PATH}/auth/login?redirect_to={original_path}"
|
||||
|
||||
logger.info(f"Unauthenticated access to {original_path}, redirecting to login")
|
||||
|
||||
return RedirectResponse(url=login_url, status_code=302)
|
||||
|
||||
# Role-based access control for specific paths
|
||||
user_roles = user.get("roles", [])
|
||||
|
||||
# /data-management/* requires 'operation' or 'admin' role
|
||||
if path.startswith("/data-management"):
|
||||
if not any(role in user_roles for role in ["admin", "operation"]):
|
||||
logger.warning(
|
||||
f"Access denied: User {user.get('username')} "
|
||||
f"(roles: {user_roles}) tried to access {path}"
|
||||
)
|
||||
# Return 403 Forbidden page
|
||||
from fastapi.responses import HTMLResponse
|
||||
return HTMLResponse(
|
||||
content=f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Access Denied</title>
|
||||
<style>
|
||||
body {{
|
||||
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
|
||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
min-height: 100vh;
|
||||
margin: 0;
|
||||
}}
|
||||
.container {{
|
||||
background: white;
|
||||
padding: 60px;
|
||||
border-radius: 20px;
|
||||
box-shadow: 0 20px 60px rgba(0,0,0,0.3);
|
||||
text-align: center;
|
||||
max-width: 500px;
|
||||
}}
|
||||
h1 {{ color: #ff6b6b; font-size: 48px; margin-bottom: 20px; }}
|
||||
p {{ color: #666; font-size: 18px; margin: 15px 0; }}
|
||||
.role-info {{
|
||||
background: #f8f9fa;
|
||||
padding: 20px;
|
||||
border-radius: 10px;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.required {{ color: #ff6b6b; font-weight: bold; }}
|
||||
.your-roles {{ color: #667eea; font-weight: bold; }}
|
||||
a {{
|
||||
display: inline-block;
|
||||
margin-top: 30px;
|
||||
padding: 12px 30px;
|
||||
background: #667eea;
|
||||
color: white;
|
||||
text-decoration: none;
|
||||
border-radius: 8px;
|
||||
font-weight: 600;
|
||||
transition: all 0.3s;
|
||||
}}
|
||||
a:hover {{ background: #5568d3; transform: translateY(-2px); }}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<h1>🚫</h1>
|
||||
<h2>Access Denied</h2>
|
||||
<p>You don't have permission to access this page.</p>
|
||||
<div class="role-info">
|
||||
<p>Required role: <span class="required">operation</span> or <span class="required">admin</span></p>
|
||||
<p>Your roles: <span class="your-roles">{', '.join(user_roles) if user_roles else 'None'}</span></p>
|
||||
</div>
|
||||
<p>Please contact your administrator if you need access.</p>
|
||||
<a href="{settings.ROOT_PATH}/">← Go to Home</a>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
""",
|
||||
status_code=403
|
||||
)
|
||||
|
||||
# Continue with request
|
||||
return await call_next(request)
|
||||
7
03-apiservice/app/models/__init__.py
Normal file
7
03-apiservice/app/models/__init__.py
Normal file
@@ -0,0 +1,7 @@
|
||||
"""
|
||||
Database models package
|
||||
"""
|
||||
from app.models.user import User, Role
|
||||
from app.models.upload import UploadHistory
|
||||
|
||||
__all__ = ["User", "Role", "UploadHistory"]
|
||||
33
03-apiservice/app/models/upload.py
Normal file
33
03-apiservice/app/models/upload.py
Normal file
@@ -0,0 +1,33 @@
|
||||
"""
|
||||
Upload History model for tracking file uploads
|
||||
"""
|
||||
from sqlalchemy import Column, Integer, String, DateTime, Text
|
||||
from sqlalchemy.sql import func
|
||||
from app.db.base import Base
|
||||
|
||||
|
||||
class UploadHistory(Base):
|
||||
"""
|
||||
Upload history tracking
|
||||
Stores information about uploaded files and their processing status
|
||||
"""
|
||||
__tablename__ = "upload_history"
|
||||
__table_args__ = {'schema': 'fastapi'}
|
||||
|
||||
id = Column(Integer, primary_key=True, index=True)
|
||||
upload_id = Column(String, unique=True, index=True, nullable=False)
|
||||
filename = Column(String, nullable=False)
|
||||
filepath = Column(String, nullable=False)
|
||||
description = Column(Text)
|
||||
status = Column(String, default="pending")
|
||||
job_id = Column(String)
|
||||
logs = Column(Text)
|
||||
uploaded_by = Column(String)
|
||||
uploaded_at = Column(DateTime(timezone=True), server_default=func.now(), nullable=False)
|
||||
updated_at = Column(DateTime(timezone=True), onupdate=func.now())
|
||||
|
||||
airflow_dag_run_id = Column(String)
|
||||
airflow_state = Column(String)
|
||||
processing_started_at = Column(DateTime(timezone=True))
|
||||
processing_completed_at = Column(DateTime(timezone=True))
|
||||
error_message = Column(Text)
|
||||
55
03-apiservice/app/models/user.py
Normal file
55
03-apiservice/app/models/user.py
Normal file
@@ -0,0 +1,55 @@
|
||||
"""
|
||||
User and Role models for local user management
|
||||
Note: This is separate from Keycloak users - used for tracking and audit
|
||||
"""
|
||||
from sqlalchemy import Column, Integer, String, Boolean, DateTime, Table, ForeignKey
|
||||
from sqlalchemy.orm import relationship
|
||||
from sqlalchemy.sql import func
|
||||
from app.db.base import Base
|
||||
|
||||
# Association table for many-to-many relationship
|
||||
user_roles = Table(
|
||||
'user_roles',
|
||||
Base.metadata,
|
||||
Column('user_id', Integer, ForeignKey('fastapi.users.id'), primary_key=True),
|
||||
Column('role_id', Integer, ForeignKey('fastapi.roles.id'), primary_key=True),
|
||||
schema='fastapi'
|
||||
)
|
||||
|
||||
|
||||
class User(Base):
|
||||
"""
|
||||
Local user record (synced from Keycloak)
|
||||
Used for tracking, audit, and local permissions
|
||||
"""
|
||||
__tablename__ = "users"
|
||||
__table_args__ = {'schema': 'fastapi'}
|
||||
|
||||
id = Column(Integer, primary_key=True, index=True)
|
||||
keycloak_id = Column(String, unique=True, index=True, nullable=False) # Keycloak sub
|
||||
username = Column(String, unique=True, index=True, nullable=False)
|
||||
email = Column(String, unique=True, index=True)
|
||||
full_name = Column(String)
|
||||
is_active = Column(Boolean, default=True)
|
||||
created_at = Column(DateTime(timezone=True), server_default=func.now())
|
||||
updated_at = Column(DateTime(timezone=True), onupdate=func.now())
|
||||
last_login = Column(DateTime(timezone=True))
|
||||
|
||||
# Relationships
|
||||
roles = relationship("Role", secondary=user_roles, back_populates="users")
|
||||
|
||||
|
||||
class Role(Base):
|
||||
"""
|
||||
Roles (synced from Keycloak)
|
||||
"""
|
||||
__tablename__ = "roles"
|
||||
__table_args__ = {'schema': 'fastapi'}
|
||||
|
||||
id = Column(Integer, primary_key=True, index=True)
|
||||
name = Column(String, unique=True, nullable=False, index=True) # admin, operation
|
||||
description = Column(String)
|
||||
created_at = Column(DateTime(timezone=True), server_default=func.now())
|
||||
|
||||
# Relationships
|
||||
users = relationship("User", secondary=user_roles, back_populates="roles")
|
||||
1
03-apiservice/app/routes/__init__.py
Normal file
1
03-apiservice/app/routes/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
# Routes package
|
||||
156
03-apiservice/app/routes/admin_users.py
Normal file
156
03-apiservice/app/routes/admin_users.py
Normal file
@@ -0,0 +1,156 @@
|
||||
"""
|
||||
User and Role management endpoints (Admin only)
|
||||
Note: This manages local user records synced from Keycloak
|
||||
API endpoints (/api/v1/*) are NOT affected and continue using API Key authentication
|
||||
"""
|
||||
from typing import List
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from sqlalchemy.orm import Session
|
||||
from pydantic import BaseModel
|
||||
from datetime import datetime
|
||||
|
||||
from app.db.session import get_db
|
||||
from app.models.user import User, Role
|
||||
from app.security.permissions import require_role, Roles
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/admin/users", tags=["admin-users"])
|
||||
|
||||
|
||||
# Pydantic schemas
|
||||
class RoleSchema(BaseModel):
|
||||
id: int
|
||||
name: str
|
||||
description: str | None = None
|
||||
|
||||
class Config:
|
||||
from_attributes = True
|
||||
|
||||
|
||||
class UserSchema(BaseModel):
|
||||
id: int
|
||||
keycloak_id: str
|
||||
username: str
|
||||
email: str | None = None
|
||||
full_name: str | None = None
|
||||
is_active: bool
|
||||
roles: List[RoleSchema] = []
|
||||
last_login: datetime | None = None
|
||||
created_at: datetime
|
||||
|
||||
class Config:
|
||||
from_attributes = True
|
||||
|
||||
|
||||
class UserCreateSchema(BaseModel):
|
||||
keycloak_id: str
|
||||
username: str
|
||||
email: str | None = None
|
||||
full_name: str | None = None
|
||||
|
||||
|
||||
class UserUpdateSchema(BaseModel):
|
||||
email: str | None = None
|
||||
full_name: str | None = None
|
||||
is_active: bool | None = None
|
||||
role_ids: List[int] | None = None
|
||||
|
||||
|
||||
@router.get("/", response_model=List[UserSchema])
|
||||
async def list_users(
|
||||
db: Session = Depends(get_db),
|
||||
current_user: dict = Depends(require_role(Roles.ADMIN))
|
||||
):
|
||||
"""List all users (Admin only)"""
|
||||
users = db.query(User).all()
|
||||
return users
|
||||
|
||||
|
||||
@router.get("/{user_id}", response_model=UserSchema)
|
||||
async def get_user(
|
||||
user_id: int,
|
||||
db: Session = Depends(get_db),
|
||||
current_user: dict = Depends(require_role(Roles.ADMIN))
|
||||
):
|
||||
"""Get user by ID (Admin only)"""
|
||||
user = db.query(User).filter(User.id == user_id).first()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
return user
|
||||
|
||||
|
||||
@router.post("/", response_model=UserSchema)
|
||||
async def create_user(
|
||||
user_data: UserCreateSchema,
|
||||
db: Session = Depends(get_db),
|
||||
current_user: dict = Depends(require_role(Roles.ADMIN))
|
||||
):
|
||||
"""Create new user record (Admin only)"""
|
||||
# Check if user already exists
|
||||
existing = db.query(User).filter(User.keycloak_id == user_data.keycloak_id).first()
|
||||
if existing:
|
||||
raise HTTPException(status_code=400, detail="User already exists")
|
||||
|
||||
user = User(**user_data.dict())
|
||||
db.add(user)
|
||||
db.commit()
|
||||
db.refresh(user)
|
||||
return user
|
||||
|
||||
|
||||
@router.put("/{user_id}", response_model=UserSchema)
|
||||
async def update_user(
|
||||
user_id: int,
|
||||
user_data: UserUpdateSchema,
|
||||
db: Session = Depends(get_db),
|
||||
current_user: dict = Depends(require_role(Roles.ADMIN))
|
||||
):
|
||||
"""Update user (Admin only)"""
|
||||
user = db.query(User).filter(User.id == user_id).first()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
# Update fields
|
||||
if user_data.email is not None:
|
||||
user.email = user_data.email
|
||||
if user_data.full_name is not None:
|
||||
user.full_name = user_data.full_name
|
||||
if user_data.is_active is not None:
|
||||
user.is_active = user_data.is_active
|
||||
|
||||
# Update roles
|
||||
if user_data.role_ids is not None:
|
||||
roles = db.query(Role).filter(Role.id.in_(user_data.role_ids)).all()
|
||||
user.roles = roles
|
||||
|
||||
db.commit()
|
||||
db.refresh(user)
|
||||
return user
|
||||
|
||||
|
||||
@router.delete("/{user_id}")
|
||||
async def delete_user(
|
||||
user_id: int,
|
||||
db: Session = Depends(get_db),
|
||||
current_user: dict = Depends(require_role(Roles.ADMIN))
|
||||
):
|
||||
"""Delete user (Admin only)"""
|
||||
user = db.query(User).filter(User.id == user_id).first()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
db.delete(user)
|
||||
db.commit()
|
||||
return {"message": "User deleted successfully"}
|
||||
|
||||
|
||||
@router.get("/roles/", response_model=List[RoleSchema])
|
||||
async def list_roles(
|
||||
db: Session = Depends(get_db),
|
||||
current_user: dict = Depends(require_role(Roles.ADMIN))
|
||||
):
|
||||
"""List all roles (Admin only)"""
|
||||
roles = db.query(Role).all()
|
||||
return roles
|
||||
252
03-apiservice/app/routes/auth.py
Normal file
252
03-apiservice/app/routes/auth.py
Normal file
@@ -0,0 +1,252 @@
|
||||
"""
|
||||
Authentication routes for Keycloak web login
|
||||
Note: These routes are ONLY for web UI authentication
|
||||
API endpoints use API Key authentication separately
|
||||
"""
|
||||
from fastapi import APIRouter, Request, HTTPException, Query
|
||||
from fastapi.responses import RedirectResponse
|
||||
from app.security.keycloak_auth import (
|
||||
get_keycloak_client,
|
||||
get_login_url,
|
||||
get_logout_url,
|
||||
get_current_user
|
||||
)
|
||||
from app.core.config import settings
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/auth", tags=["authentication"])
|
||||
|
||||
|
||||
@router.get("/login")
|
||||
async def login(request: Request, redirect_to: str = Query(default="/")):
|
||||
"""
|
||||
Redirect to Keycloak login page
|
||||
|
||||
Args:
|
||||
redirect_to: Path to redirect after successful login
|
||||
"""
|
||||
# Check if already logged in
|
||||
user = get_current_user(request)
|
||||
if user:
|
||||
return RedirectResponse(url=redirect_to)
|
||||
|
||||
# Generate Keycloak login URL
|
||||
login_url = get_login_url(redirect_to)
|
||||
return RedirectResponse(url=login_url)
|
||||
|
||||
|
||||
@router.get("/callback")
|
||||
async def auth_callback(
|
||||
request: Request,
|
||||
code: str = Query(...),
|
||||
state: str = Query(default="/")
|
||||
):
|
||||
"""
|
||||
Handle Keycloak callback after login
|
||||
|
||||
Args:
|
||||
code: Authorization code from Keycloak
|
||||
state: Original redirect path
|
||||
"""
|
||||
try:
|
||||
if settings.DEBUG_AUTH:
|
||||
logger.info("=" * 80)
|
||||
logger.info("AUTHENTICATION CALLBACK RECEIVED")
|
||||
logger.info(f"Authorization code: {code[:20]}...{code[-20:] if len(code) > 40 else code}")
|
||||
logger.info(f"State (redirect_to): {state}")
|
||||
logger.info(f"Request URL: {request.url}")
|
||||
logger.info(f"Request headers: {dict(request.headers)}")
|
||||
|
||||
keycloak_client = get_keycloak_client()
|
||||
|
||||
if settings.DEBUG_AUTH:
|
||||
logger.info("-" * 80)
|
||||
logger.info("EXCHANGING CODE FOR TOKENS")
|
||||
logger.info(f"Grant type: authorization_code")
|
||||
logger.info(f"Code: {code[:20]}...")
|
||||
logger.info(f"Redirect URI: {settings.KEYCLOAK_REDIRECT_URI}")
|
||||
|
||||
# Exchange authorization code for tokens
|
||||
token_response = keycloak_client.token(
|
||||
grant_type="authorization_code",
|
||||
code=code,
|
||||
redirect_uri=settings.KEYCLOAK_REDIRECT_URI
|
||||
)
|
||||
|
||||
if settings.DEBUG_AUTH:
|
||||
logger.info("TOKEN RESPONSE RECEIVED")
|
||||
logger.info(f"Access token: {'*' * 20}...{token_response.get('access_token', '')[-20:] if token_response.get('access_token') else 'NONE'}")
|
||||
logger.info(f"Refresh token: {'Present' if token_response.get('refresh_token') else 'None'}")
|
||||
logger.info(f"Token type: {token_response.get('token_type', 'N/A')}")
|
||||
logger.info(f"Expires in: {token_response.get('expires_in', 'N/A')} seconds")
|
||||
|
||||
# Get user information from token
|
||||
access_token = token_response.get("access_token")
|
||||
if not access_token:
|
||||
logger.error("No access token in response!")
|
||||
if settings.DEBUG_AUTH:
|
||||
logger.error(f"Full token response: {token_response}")
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="No access token received from Keycloak"
|
||||
)
|
||||
|
||||
if settings.DEBUG_AUTH:
|
||||
logger.info("-" * 80)
|
||||
logger.info("FETCHING USER INFO")
|
||||
|
||||
from jose import jwt
|
||||
userinfo = jwt.decode(access_token, key="", options={"verify_signature": False, "verify_aud": False})
|
||||
logger.info(f"Decoded access_token: {userinfo.get('preferred_username')}")
|
||||
|
||||
# # 2. ดึง id_token ออกมา (ตัวนี้คือหัวใจของ OIDC)
|
||||
# id_token = token_response.get("id_token")
|
||||
|
||||
# if not id_token:
|
||||
# logger.error("No id_token in response!")
|
||||
# if settings.DEBUG_AUTH:
|
||||
# logger.error(f"Full token response: {token_response}")
|
||||
# raise HTTPException(
|
||||
# status_code=400,
|
||||
# detail="No id_token received from Keycloak"
|
||||
# )
|
||||
|
||||
# # 3. Decode id_token เพื่อเอาข้อมูล User (ไม่ต้องใช้ Key เพราะเราเชื่อถือ Connection นี้)
|
||||
# userinfo = jwt.decode(
|
||||
# id_token,
|
||||
# key="",
|
||||
# options={"verify_signature": False, "verify_aud": False}
|
||||
# )
|
||||
|
||||
#userinfo = keycloak_client.userinfo(access_token)
|
||||
|
||||
# Extract roles from token
|
||||
roles = []
|
||||
|
||||
# 1. Realm roles (roles ระดับ realm)
|
||||
if "realm_access" in userinfo and "roles" in userinfo["realm_access"]:
|
||||
roles.extend(userinfo["realm_access"]["roles"])
|
||||
|
||||
# 2. Client roles (roles เฉพาะ client apiservice)
|
||||
if "resource_access" in userinfo and settings.KEYCLOAK_CLIENT_ID in userinfo["resource_access"]:
|
||||
client_roles = userinfo["resource_access"][settings.KEYCLOAK_CLIENT_ID].get("roles", [])
|
||||
roles.extend(client_roles)
|
||||
|
||||
# Filter to only include our application roles
|
||||
user_roles = [r for r in roles if r in ["admin", "operation"]]
|
||||
|
||||
if settings.DEBUG_AUTH:
|
||||
logger.info("USER INFO RECEIVED")
|
||||
logger.info(f"Username: {userinfo.get('preferred_username')}")
|
||||
logger.info(f"Email: {userinfo.get('email')}")
|
||||
logger.info(f"Name: {userinfo.get('name')}")
|
||||
logger.info(f"Sub (User ID): {userinfo.get('sub')}")
|
||||
logger.info(f"All roles from token: {roles}")
|
||||
logger.info(f"Filtered user roles: {user_roles}")
|
||||
logger.info(f"Full userinfo keys: {list(userinfo.keys())}")
|
||||
|
||||
# Store user info, roles, and tokens in session
|
||||
user_session_data = {
|
||||
"username": userinfo.get("preferred_username"),
|
||||
"email": userinfo.get("email"),
|
||||
"name": userinfo.get("name", userinfo.get("preferred_username")),
|
||||
"sub": userinfo.get("sub"), # User ID
|
||||
"roles": user_roles, # User roles
|
||||
"access_token": access_token,
|
||||
"refresh_token": token_response.get("refresh_token")
|
||||
}
|
||||
|
||||
request.session["user"] = user_session_data
|
||||
|
||||
if settings.DEBUG_AUTH:
|
||||
logger.info("-" * 80)
|
||||
logger.info("SESSION UPDATED")
|
||||
logger.info(f"Session user data: {dict((k, v) for k, v in user_session_data.items() if k not in ['access_token', 'refresh_token'])}")
|
||||
|
||||
logger.info(f"User {userinfo.get('preferred_username')} logged in successfully")
|
||||
|
||||
# Redirect to original destination
|
||||
redirect_url = state if state else "/"
|
||||
|
||||
# Ensure redirect URL starts with root_path if set
|
||||
if settings.ROOT_PATH and not redirect_url.startswith(settings.ROOT_PATH):
|
||||
redirect_url = f"{settings.ROOT_PATH}{redirect_url}"
|
||||
|
||||
if settings.DEBUG_AUTH:
|
||||
logger.info(f"Redirecting to: {redirect_url}")
|
||||
logger.info("=" * 80)
|
||||
|
||||
return RedirectResponse(url=redirect_url, status_code=302)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Authentication callback failed: {e}")
|
||||
if settings.DEBUG_AUTH:
|
||||
import traceback
|
||||
logger.error("FULL TRACEBACK:")
|
||||
logger.error(traceback.format_exc())
|
||||
logger.error("=" * 80)
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Authentication failed: {str(e)}"
|
||||
)
|
||||
|
||||
|
||||
@router.get("/logout")
|
||||
async def logout(request: Request):
|
||||
"""
|
||||
Logout user and clear session
|
||||
Redirects to Keycloak logout page
|
||||
"""
|
||||
user = get_current_user(request)
|
||||
|
||||
# Clear session
|
||||
request.session.clear()
|
||||
|
||||
if user:
|
||||
logger.info(f"User {user.get('username')} logged out")
|
||||
|
||||
# Get Keycloak logout URL
|
||||
redirect_uri = f"{settings.ROOT_PATH}/" if settings.ROOT_PATH else "/"
|
||||
logout_url = get_logout_url(redirect_uri)
|
||||
|
||||
return RedirectResponse(url=logout_url)
|
||||
|
||||
|
||||
@router.get("/user")
|
||||
async def get_user_info(request: Request):
|
||||
"""
|
||||
Get current authenticated user information
|
||||
Returns 401 if not authenticated
|
||||
"""
|
||||
user = get_current_user(request)
|
||||
if not user:
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Not authenticated"
|
||||
)
|
||||
|
||||
# Return user info without sensitive tokens
|
||||
return {
|
||||
"username": user.get("username"),
|
||||
"email": user.get("email"),
|
||||
"name": user.get("name"),
|
||||
"sub": user.get("sub")
|
||||
}
|
||||
|
||||
|
||||
@router.get("/status")
|
||||
async def auth_status(request: Request):
|
||||
"""
|
||||
Check authentication status
|
||||
Returns whether user is logged in
|
||||
"""
|
||||
user = get_current_user(request)
|
||||
return {
|
||||
"authenticated": user is not None,
|
||||
"user": {
|
||||
"username": user.get("username"),
|
||||
"name": user.get("name")
|
||||
} if user else None
|
||||
}
|
||||
305
03-apiservice/app/routes/pages.py
Normal file
305
03-apiservice/app/routes/pages.py
Normal file
@@ -0,0 +1,305 @@
|
||||
"""
|
||||
Web page routes for the application
|
||||
"""
|
||||
import os
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import List
|
||||
|
||||
from fastapi import APIRouter, Request, UploadFile, File, Form, HTTPException, Depends
|
||||
from fastapi.responses import HTMLResponse
|
||||
from fastapi.templating import Jinja2Templates
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy.orm import Session
|
||||
import asyncio
|
||||
import logging
|
||||
|
||||
from app.core.config import settings
|
||||
from app.security.permissions import require_role, Roles
|
||||
from app.db.session import get_db
|
||||
from app.models.upload import UploadHistory
|
||||
from app.services.airflow_client import airflow_client
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
# Setup templates
|
||||
templates_dir = Path(__file__).parent.parent / "templates"
|
||||
templates = Jinja2Templates(directory=str(templates_dir))
|
||||
|
||||
# Upload directory
|
||||
UPLOAD_DIR = Path("/data/uploads")
|
||||
UPLOAD_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
class UploadRecordSchema(BaseModel):
|
||||
id: int
|
||||
upload_id: str
|
||||
filename: str
|
||||
filepath: str
|
||||
description: str | None = None
|
||||
status: str
|
||||
job_id: str | None = None
|
||||
logs: str | None = None
|
||||
uploaded_by: str | None = None
|
||||
uploaded_at: datetime
|
||||
updated_at: datetime | None = None
|
||||
|
||||
class Config:
|
||||
from_attributes = True
|
||||
|
||||
|
||||
@router.get("/", response_class=HTMLResponse)
|
||||
async def index(request: Request):
|
||||
"""Landing page with navigation menu"""
|
||||
user = request.session.get("user")
|
||||
return templates.TemplateResponse(
|
||||
"index.html",
|
||||
{
|
||||
"request": request,
|
||||
"root_path": settings.ROOT_PATH,
|
||||
"user": user
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@router.get("/data-management/finance", response_class=HTMLResponse)
|
||||
async def finance_page(request: Request):
|
||||
"""Finance Excel upload page - requires operation or admin role"""
|
||||
user = request.session.get("user")
|
||||
return templates.TemplateResponse(
|
||||
"data_management_finance.html",
|
||||
{
|
||||
"request": request,
|
||||
"root_path": settings.ROOT_PATH,
|
||||
"user": user
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@router.get("/admin/users", response_class=HTMLResponse)
|
||||
async def admin_users_page(
|
||||
request: Request,
|
||||
current_user: dict = Depends(require_role(Roles.ADMIN))
|
||||
):
|
||||
"""User management page - Admin only"""
|
||||
return templates.TemplateResponse(
|
||||
"admin_users.html",
|
||||
{
|
||||
"request": request,
|
||||
"root_path": settings.ROOT_PATH,
|
||||
"user": current_user
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@router.post("/data-management/finance/upload")
|
||||
async def upload_finance_file(
|
||||
request: Request,
|
||||
file: UploadFile = File(...),
|
||||
description: str = Form(None),
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
"""
|
||||
Handle finance Excel file upload
|
||||
|
||||
- Saves file to /data/uploads/
|
||||
- Stores upload record in database
|
||||
- Triggers Airflow job (to be implemented)
|
||||
- Returns upload record
|
||||
"""
|
||||
# Validate file type
|
||||
if not file.filename.endswith(('.xlsx', '.xls')):
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="Invalid file type. Only .xlsx and .xls files are allowed."
|
||||
)
|
||||
|
||||
# Generate unique filename
|
||||
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
safe_filename = file.filename.replace(" ", "_")
|
||||
unique_filename = f"{timestamp}_{safe_filename}"
|
||||
filepath = UPLOAD_DIR / unique_filename
|
||||
|
||||
# Save file
|
||||
try:
|
||||
content = await file.read()
|
||||
with open(filepath, "wb") as f:
|
||||
f.write(content)
|
||||
except Exception as e:
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"Failed to save file: {str(e)}"
|
||||
)
|
||||
|
||||
# Get username from session
|
||||
user = request.session.get("user")
|
||||
username = user.get("username") if user else "anonymous"
|
||||
|
||||
# Create upload record in database
|
||||
upload_id = f"upload_{timestamp}"
|
||||
upload_record = UploadHistory(
|
||||
upload_id=upload_id,
|
||||
filename=file.filename,
|
||||
filepath=str(filepath),
|
||||
description=description,
|
||||
status="pending",
|
||||
uploaded_by=username
|
||||
)
|
||||
|
||||
db.add(upload_record)
|
||||
db.commit()
|
||||
db.refresh(upload_record)
|
||||
|
||||
# Trigger Airflow DAG with retry logic
|
||||
airflow_triggered = False
|
||||
dag_run_id = None
|
||||
error_msg = None
|
||||
|
||||
max_retries = 3
|
||||
retry_delay = 10 # seconds
|
||||
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
logger.info(f"Triggering Airflow DAG (attempt {attempt + 1}/{max_retries})")
|
||||
|
||||
result = await airflow_client.trigger_finance_dag(
|
||||
upload_id=upload_id,
|
||||
filepath=str(filepath),
|
||||
filename=file.filename,
|
||||
uploaded_by=username,
|
||||
description=description
|
||||
)
|
||||
|
||||
dag_run_id = result.get("dag_run_id")
|
||||
airflow_triggered = True
|
||||
|
||||
# Update upload record with Airflow info
|
||||
upload_record.airflow_dag_run_id = dag_run_id
|
||||
upload_record.airflow_state = result.get("state", "queued")
|
||||
upload_record.status = "processing"
|
||||
db.commit()
|
||||
|
||||
logger.info(f"Airflow DAG triggered successfully: {dag_run_id}")
|
||||
break
|
||||
|
||||
except Exception as e:
|
||||
error_msg = str(e)
|
||||
logger.error(f"Failed to trigger Airflow (attempt {attempt + 1}/{max_retries}): {error_msg}")
|
||||
|
||||
if attempt < max_retries - 1:
|
||||
logger.info(f"Retrying in {retry_delay} seconds...")
|
||||
await asyncio.sleep(retry_delay)
|
||||
else:
|
||||
logger.error(f"All {max_retries} attempts failed to trigger Airflow")
|
||||
upload_record.status = "error"
|
||||
upload_record.error_message = f"Failed to trigger Airflow after {max_retries} attempts: {error_msg}"
|
||||
db.commit()
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"message": f"File '{file.filename}' uploaded successfully",
|
||||
"upload_id": upload_id,
|
||||
"filename": unique_filename,
|
||||
"airflow_triggered": airflow_triggered,
|
||||
"dag_run_id": dag_run_id,
|
||||
"error": error_msg if not airflow_triggered else None
|
||||
}
|
||||
|
||||
|
||||
@router.get("/data-management/finance/uploads")
|
||||
async def get_uploads(db: Session = Depends(get_db)):
|
||||
"""Get list of all uploads with their status"""
|
||||
uploads = db.query(UploadHistory).order_by(UploadHistory.uploaded_at.desc()).all()
|
||||
|
||||
# Convert to dict for JSON response
|
||||
return [
|
||||
{
|
||||
"id": upload.upload_id,
|
||||
"filename": upload.filename,
|
||||
"filepath": upload.filepath,
|
||||
"uploaded_at": upload.uploaded_at.isoformat(),
|
||||
"description": upload.description,
|
||||
"status": upload.status,
|
||||
"job_id": upload.job_id,
|
||||
"logs": upload.logs,
|
||||
"uploaded_by": upload.uploaded_by,
|
||||
"airflow_dag_run_id": upload.airflow_dag_run_id,
|
||||
"airflow_state": upload.airflow_state,
|
||||
"processing_started_at": upload.processing_started_at.isoformat() if upload.processing_started_at else None,
|
||||
"processing_completed_at": upload.processing_completed_at.isoformat() if upload.processing_completed_at else None,
|
||||
"error_message": upload.error_message
|
||||
}
|
||||
for upload in uploads
|
||||
]
|
||||
|
||||
|
||||
@router.get("/data-management/finance/uploads/{upload_id}")
|
||||
async def get_upload_status(upload_id: str, db: Session = Depends(get_db)):
|
||||
"""Get status of a specific upload"""
|
||||
upload = db.query(UploadHistory).filter(UploadHistory.upload_id == upload_id).first()
|
||||
if not upload:
|
||||
raise HTTPException(status_code=404, detail="Upload not found")
|
||||
|
||||
return {
|
||||
"id": upload.upload_id,
|
||||
"filename": upload.filename,
|
||||
"filepath": upload.filepath,
|
||||
"uploaded_at": upload.uploaded_at.isoformat(),
|
||||
"description": upload.description,
|
||||
"status": upload.status,
|
||||
"job_id": upload.job_id,
|
||||
"logs": upload.logs,
|
||||
"uploaded_by": upload.uploaded_by,
|
||||
"airflow_dag_run_id": upload.airflow_dag_run_id,
|
||||
"airflow_state": upload.airflow_state,
|
||||
"processing_started_at": upload.processing_started_at.isoformat() if upload.processing_started_at else None,
|
||||
"processing_completed_at": upload.processing_completed_at.isoformat() if upload.processing_completed_at else None,
|
||||
"error_message": upload.error_message
|
||||
}
|
||||
|
||||
|
||||
# Placeholder for Airflow integration
|
||||
async def trigger_airflow_job(filepath: str, upload_id: str) -> str:
|
||||
"""
|
||||
Trigger Airflow DAG to process the uploaded file
|
||||
|
||||
Args:
|
||||
filepath: Path to the uploaded file
|
||||
upload_id: Unique upload identifier
|
||||
|
||||
Returns:
|
||||
job_id: Airflow job/run ID
|
||||
|
||||
This function will be implemented when:
|
||||
- Airflow DAG ID is provided
|
||||
- Airflow API endpoint is configured
|
||||
"""
|
||||
# TODO: Implement Airflow API call
|
||||
# Example implementation:
|
||||
# import httpx
|
||||
#
|
||||
# airflow_url = "http://airflow-webserver:8080/api/v1/dags/{dag_id}/dagRuns"
|
||||
# headers = {"Content-Type": "application/json"}
|
||||
# auth = ("airflow", "airflow") # Use proper credentials
|
||||
#
|
||||
# payload = {
|
||||
# "conf": {
|
||||
# "filepath": filepath,
|
||||
# "upload_id": upload_id
|
||||
# }
|
||||
# }
|
||||
#
|
||||
# async with httpx.AsyncClient() as client:
|
||||
# response = await client.post(
|
||||
# airflow_url,
|
||||
# json=payload,
|
||||
# headers=headers,
|
||||
# auth=auth
|
||||
# )
|
||||
# response.raise_for_status()
|
||||
# result = response.json()
|
||||
# return result["dag_run_id"]
|
||||
|
||||
raise NotImplementedError("Airflow integration pending DAG ID and endpoint")
|
||||
146
03-apiservice/app/security/keycloak_auth.py
Normal file
146
03-apiservice/app/security/keycloak_auth.py
Normal file
@@ -0,0 +1,146 @@
|
||||
"""
|
||||
Keycloak authentication for web pages
|
||||
Note: This is ONLY for web UI authentication, NOT for API endpoints
|
||||
API endpoints use API Key authentication (see app/security/dependencies.py)
|
||||
"""
|
||||
from typing import Optional
|
||||
from fastapi import HTTPException, Request, status
|
||||
from keycloak import KeycloakOpenID
|
||||
from app.core.config import settings
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Initialize Keycloak OpenID client
|
||||
def get_keycloak_client() -> KeycloakOpenID:
|
||||
"""Get Keycloak OpenID client instance"""
|
||||
try:
|
||||
if settings.DEBUG_AUTH:
|
||||
logger.info("=" * 80)
|
||||
logger.info("KEYCLOAK CLIENT INITIALIZATION")
|
||||
logger.info(f"Server URL: {settings.KEYCLOAK_SERVER_URL}")
|
||||
logger.info(f"Realm: {settings.KEYCLOAK_REALM}")
|
||||
logger.info(f"Client ID: {settings.KEYCLOAK_CLIENT_ID}")
|
||||
logger.info(f"Client Secret: {'*' * len(settings.KEYCLOAK_CLIENT_SECRET) if settings.KEYCLOAK_CLIENT_SECRET else 'NOT SET'}")
|
||||
logger.info(f"Redirect URI: {settings.KEYCLOAK_REDIRECT_URI}")
|
||||
logger.info("=" * 80)
|
||||
|
||||
return KeycloakOpenID(
|
||||
server_url=settings.KEYCLOAK_SERVER_URL,
|
||||
client_id=settings.KEYCLOAK_CLIENT_ID,
|
||||
realm_name=settings.KEYCLOAK_REALM,
|
||||
client_secret_key=settings.KEYCLOAK_CLIENT_SECRET,
|
||||
verify=True
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to initialize Keycloak client: {e}")
|
||||
if settings.DEBUG_AUTH:
|
||||
import traceback
|
||||
logger.error(f"Traceback: {traceback.format_exc()}")
|
||||
raise
|
||||
|
||||
|
||||
def get_current_user(request: Request) -> Optional[dict]:
|
||||
"""
|
||||
Get current authenticated user from session
|
||||
Returns None if not authenticated
|
||||
"""
|
||||
return request.session.get("user")
|
||||
|
||||
|
||||
def require_user(request: Request) -> dict:
|
||||
"""
|
||||
Dependency to require authenticated user
|
||||
Raises 401 if not authenticated
|
||||
"""
|
||||
user = get_current_user(request)
|
||||
if not user:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Authentication required"
|
||||
)
|
||||
return user
|
||||
|
||||
|
||||
def get_login_url(redirect_to: str = "/") -> str:
|
||||
"""
|
||||
Generate Keycloak login URL
|
||||
|
||||
Args:
|
||||
redirect_to: Path to redirect after successful login
|
||||
|
||||
Returns:
|
||||
Keycloak authorization URL
|
||||
"""
|
||||
try:
|
||||
if settings.DEBUG_AUTH:
|
||||
logger.info("=" * 80)
|
||||
logger.info("GENERATING LOGIN URL")
|
||||
logger.info(f"Redirect to after login: {redirect_to}")
|
||||
logger.info(f"Keycloak redirect URI: {settings.KEYCLOAK_REDIRECT_URI}")
|
||||
|
||||
keycloak_client = get_keycloak_client()
|
||||
auth_url = keycloak_client.auth_url(
|
||||
redirect_uri=settings.KEYCLOAK_REDIRECT_URI,
|
||||
state=redirect_to
|
||||
)
|
||||
|
||||
if settings.DEBUG_AUTH:
|
||||
logger.info(f"Generated auth URL: {auth_url}")
|
||||
logger.info("=" * 80)
|
||||
|
||||
return auth_url
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to generate login URL: {e}")
|
||||
if settings.DEBUG_AUTH:
|
||||
import traceback
|
||||
logger.error(f"Traceback: {traceback.format_exc()}")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail="Authentication service unavailable"
|
||||
)
|
||||
|
||||
|
||||
async def verify_token(token: str) -> dict:
|
||||
"""
|
||||
Verify and decode Keycloak access token
|
||||
|
||||
Args:
|
||||
token: Access token from Keycloak
|
||||
|
||||
Returns:
|
||||
User information from token
|
||||
"""
|
||||
try:
|
||||
keycloak_client = get_keycloak_client()
|
||||
userinfo = keycloak_client.userinfo(token)
|
||||
return userinfo
|
||||
except Exception as e:
|
||||
logger.error(f"Token verification failed: {e}")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Invalid or expired token"
|
||||
)
|
||||
|
||||
|
||||
def get_logout_url(redirect_uri: str = None) -> str:
|
||||
"""
|
||||
Generate Keycloak logout URL
|
||||
|
||||
Args:
|
||||
redirect_uri: Where to redirect after logout
|
||||
|
||||
Returns:
|
||||
Keycloak logout URL
|
||||
"""
|
||||
try:
|
||||
keycloak_client = get_keycloak_client()
|
||||
if redirect_uri is None:
|
||||
redirect_uri = f"{settings.ROOT_PATH}/" if settings.ROOT_PATH else "/"
|
||||
|
||||
logout_url = keycloak_client.logout_url(redirect_uri=redirect_uri)
|
||||
return logout_url
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to generate logout URL: {e}")
|
||||
# Return simple redirect if Keycloak logout fails
|
||||
return redirect_uri or "/"
|
||||
151
03-apiservice/app/security/permissions.py
Normal file
151
03-apiservice/app/security/permissions.py
Normal file
@@ -0,0 +1,151 @@
|
||||
"""
|
||||
Role-based permission system for web pages
|
||||
Note: API endpoints (/api/v1/*) use API Key authentication and are not affected by this
|
||||
"""
|
||||
from typing import List
|
||||
from fastapi import HTTPException, Request, status
|
||||
from app.core.config import settings
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class Roles:
|
||||
"""Role constants"""
|
||||
ADMIN = "admin"
|
||||
OPERATION = "operation"
|
||||
|
||||
|
||||
def get_user_roles(request: Request) -> List[str]:
|
||||
"""
|
||||
Get roles from session
|
||||
|
||||
Args:
|
||||
request: FastAPI request object
|
||||
|
||||
Returns:
|
||||
List of role names
|
||||
"""
|
||||
user = request.session.get("user")
|
||||
if not user:
|
||||
return []
|
||||
return user.get("roles", [])
|
||||
|
||||
|
||||
def has_role(request: Request, required_role: str) -> bool:
|
||||
"""
|
||||
Check if user has specific role
|
||||
|
||||
Args:
|
||||
request: FastAPI request object
|
||||
required_role: Role name to check
|
||||
|
||||
Returns:
|
||||
True if user has the role, False otherwise
|
||||
"""
|
||||
user_roles = get_user_roles(request)
|
||||
return required_role in user_roles
|
||||
|
||||
|
||||
def has_any_role(request: Request, required_roles: List[str]) -> bool:
|
||||
"""
|
||||
Check if user has any of the required roles
|
||||
|
||||
Args:
|
||||
request: FastAPI request object
|
||||
required_roles: List of role names
|
||||
|
||||
Returns:
|
||||
True if user has at least one of the roles, False otherwise
|
||||
"""
|
||||
user_roles = get_user_roles(request)
|
||||
return any(role in user_roles for role in required_roles)
|
||||
|
||||
|
||||
def require_role(required_role: str):
|
||||
"""
|
||||
Dependency to require specific role
|
||||
|
||||
Args:
|
||||
required_role: Role name required to access the endpoint
|
||||
|
||||
Returns:
|
||||
Dependency function that checks for the role
|
||||
|
||||
Raises:
|
||||
HTTPException: 401 if not authenticated, 403 if missing required role
|
||||
"""
|
||||
def check_role(request: Request):
|
||||
user = request.session.get("user")
|
||||
if not user:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Authentication required"
|
||||
)
|
||||
|
||||
user_roles = user.get("roles", [])
|
||||
|
||||
if required_role not in user_roles:
|
||||
logger.warning(
|
||||
f"Access denied: User {user.get('username')} "
|
||||
f"(roles: {user_roles}) tried to access resource requiring '{required_role}'"
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_403_FORBIDDEN,
|
||||
detail=f"Access denied. Required role: {required_role}"
|
||||
)
|
||||
|
||||
return user
|
||||
|
||||
return check_role
|
||||
|
||||
|
||||
def require_any_role(required_roles: List[str]):
|
||||
"""
|
||||
Dependency to require any of the specified roles
|
||||
|
||||
Args:
|
||||
required_roles: List of role names, user needs at least one
|
||||
|
||||
Returns:
|
||||
Dependency function that checks for roles
|
||||
|
||||
Raises:
|
||||
HTTPException: 401 if not authenticated, 403 if missing all required roles
|
||||
"""
|
||||
def check_roles(request: Request):
|
||||
user = request.session.get("user")
|
||||
if not user:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Authentication required"
|
||||
)
|
||||
|
||||
user_roles = user.get("roles", [])
|
||||
|
||||
if not any(role in user_roles for role in required_roles):
|
||||
logger.warning(
|
||||
f"Access denied: User {user.get('username')} "
|
||||
f"(roles: {user_roles}) tried to access resource requiring one of: {required_roles}"
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_403_FORBIDDEN,
|
||||
detail=f"Access denied. Required roles: {', '.join(required_roles)}"
|
||||
)
|
||||
|
||||
return user
|
||||
|
||||
return check_roles
|
||||
|
||||
|
||||
def is_admin(request: Request) -> bool:
|
||||
"""
|
||||
Check if user is admin
|
||||
|
||||
Args:
|
||||
request: FastAPI request object
|
||||
|
||||
Returns:
|
||||
True if user has admin role, False otherwise
|
||||
"""
|
||||
return has_role(request, Roles.ADMIN)
|
||||
152
03-apiservice/app/services/airflow_client.py
Normal file
152
03-apiservice/app/services/airflow_client.py
Normal file
@@ -0,0 +1,152 @@
|
||||
"""
|
||||
Airflow API Client for triggering DAGs
|
||||
"""
|
||||
import httpx
|
||||
import logging
|
||||
from typing import Dict, Any, Optional
|
||||
from datetime import datetime
|
||||
|
||||
from app.core.config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class AirflowClient:
|
||||
"""Client for interacting with Airflow REST API"""
|
||||
|
||||
def __init__(self):
|
||||
self.api_url = settings.AIRFLOW_API_URL.rstrip('/')
|
||||
self.api_token = settings.AIRFLOW_API_TOKEN
|
||||
self.headers = {
|
||||
"Content-Type": "application/json",
|
||||
"Accept": "application/json"
|
||||
}
|
||||
|
||||
if self.api_token:
|
||||
self.headers["Authorization"] = f"Bearer {self.api_token}"
|
||||
else:
|
||||
logger.warning("AIRFLOW_API_TOKEN not set - API calls may fail")
|
||||
|
||||
async def trigger_dag(
|
||||
self,
|
||||
dag_id: str,
|
||||
conf: Dict[str, Any],
|
||||
logical_date: Optional[str] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Trigger an Airflow DAG run
|
||||
|
||||
Args:
|
||||
dag_id: DAG identifier
|
||||
conf: Configuration to pass to the DAG (JSON serializable dict)
|
||||
logical_date: Optional logical date for the DAG run (ISO format)
|
||||
|
||||
Returns:
|
||||
Dict containing dag_run_id and other metadata
|
||||
|
||||
Raises:
|
||||
httpx.HTTPError: If the API call fails
|
||||
"""
|
||||
url = f"{self.api_url}/api/v1/dags/{dag_id}/dagRuns"
|
||||
|
||||
payload = {
|
||||
"conf": conf
|
||||
}
|
||||
|
||||
if logical_date:
|
||||
payload["logical_date"] = logical_date
|
||||
else:
|
||||
payload["logical_date"] = datetime.utcnow().isoformat() + "Z"
|
||||
|
||||
logger.info(f"Triggering Airflow DAG: {dag_id}")
|
||||
logger.debug(f"Payload: {payload}")
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||
response = await client.post(
|
||||
url,
|
||||
json=payload,
|
||||
headers=self.headers
|
||||
)
|
||||
|
||||
response.raise_for_status()
|
||||
result = response.json()
|
||||
|
||||
logger.info(f"DAG triggered successfully: {result.get('dag_run_id')}")
|
||||
return result
|
||||
|
||||
except httpx.HTTPStatusError as e:
|
||||
logger.error(f"HTTP error triggering DAG {dag_id}: {e.response.status_code} - {e.response.text}")
|
||||
raise
|
||||
except httpx.RequestError as e:
|
||||
logger.error(f"Request error triggering DAG {dag_id}: {str(e)}")
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error(f"Unexpected error triggering DAG {dag_id}: {str(e)}")
|
||||
raise
|
||||
|
||||
async def get_dag_run_status(
|
||||
self,
|
||||
dag_id: str,
|
||||
dag_run_id: str
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Get the status of a DAG run
|
||||
|
||||
Args:
|
||||
dag_id: DAG identifier
|
||||
dag_run_id: DAG run identifier
|
||||
|
||||
Returns:
|
||||
Dict containing DAG run status and metadata
|
||||
"""
|
||||
url = f"{self.api_url}/api/v1/dags/{dag_id}/dagRuns/{dag_run_id}"
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||
response = await client.get(url, headers=self.headers)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
except httpx.HTTPError as e:
|
||||
logger.error(f"Error getting DAG run status: {str(e)}")
|
||||
raise
|
||||
|
||||
async def trigger_finance_dag(
|
||||
self,
|
||||
upload_id: str,
|
||||
filepath: str,
|
||||
filename: str,
|
||||
uploaded_by: str,
|
||||
description: Optional[str] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Trigger the finance Excel processing DAG
|
||||
|
||||
Args:
|
||||
upload_id: Unique upload identifier
|
||||
filepath: Full path to the uploaded file
|
||||
filename: Original filename
|
||||
uploaded_by: Username who uploaded the file
|
||||
description: Optional description
|
||||
|
||||
Returns:
|
||||
Dict containing dag_run_id and other metadata
|
||||
"""
|
||||
conf = {
|
||||
"upload_id": upload_id,
|
||||
"filepath": filepath,
|
||||
"filename": filename,
|
||||
"uploaded_by": uploaded_by,
|
||||
"description": description or "",
|
||||
"triggered_at": datetime.utcnow().isoformat()
|
||||
}
|
||||
|
||||
return await self.trigger_dag(
|
||||
dag_id=settings.AIRFLOW_DAG_ID_FINANCE,
|
||||
conf=conf
|
||||
)
|
||||
|
||||
|
||||
# Singleton instance
|
||||
airflow_client = AirflowClient()
|
||||
360
03-apiservice/app/templates/admin_users.html
Normal file
360
03-apiservice/app/templates/admin_users.html
Normal file
@@ -0,0 +1,360 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>User Management - Admin</title>
|
||||
<style>
|
||||
* { margin: 0; padding: 0; box-sizing: border-box; }
|
||||
body {
|
||||
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
|
||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
||||
min-height: 100vh;
|
||||
padding: 20px;
|
||||
}
|
||||
.container {
|
||||
max-width: 1400px;
|
||||
margin: 0 auto;
|
||||
background: white;
|
||||
border-radius: 20px;
|
||||
padding: 40px;
|
||||
box-shadow: 0 20px 60px rgba(0,0,0,0.3);
|
||||
}
|
||||
.header {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
margin-bottom: 30px;
|
||||
padding-bottom: 20px;
|
||||
border-bottom: 2px solid #e0e0e0;
|
||||
}
|
||||
h1 { color: #333; font-size: 32px; }
|
||||
.user-info {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 15px;
|
||||
}
|
||||
.role-badge {
|
||||
padding: 5px 15px;
|
||||
border-radius: 20px;
|
||||
font-size: 12px;
|
||||
font-weight: bold;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
.role-admin {
|
||||
background: linear-gradient(135deg, #ff6b6b 0%, #ee5a52 100%);
|
||||
color: white;
|
||||
}
|
||||
.role-operation {
|
||||
background: linear-gradient(135deg, #4ecdc4 0%, #44a08d 100%);
|
||||
color: white;
|
||||
}
|
||||
.back-link {
|
||||
display: inline-block;
|
||||
margin-bottom: 20px;
|
||||
color: #667eea;
|
||||
text-decoration: none;
|
||||
font-weight: 600;
|
||||
font-size: 16px;
|
||||
}
|
||||
.back-link:hover { text-decoration: underline; }
|
||||
|
||||
.alert {
|
||||
padding: 15px 20px;
|
||||
margin-bottom: 20px;
|
||||
border-radius: 8px;
|
||||
font-weight: 500;
|
||||
}
|
||||
.alert-success {
|
||||
background: #51cf66;
|
||||
color: white;
|
||||
}
|
||||
.alert-error {
|
||||
background: #ff6b6b;
|
||||
color: white;
|
||||
}
|
||||
|
||||
.stats {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
|
||||
gap: 20px;
|
||||
margin-bottom: 30px;
|
||||
}
|
||||
.stat-card {
|
||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
||||
color: white;
|
||||
padding: 20px;
|
||||
border-radius: 12px;
|
||||
text-align: center;
|
||||
}
|
||||
.stat-number {
|
||||
font-size: 36px;
|
||||
font-weight: bold;
|
||||
margin-bottom: 5px;
|
||||
}
|
||||
.stat-label {
|
||||
font-size: 14px;
|
||||
opacity: 0.9;
|
||||
}
|
||||
|
||||
table {
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
margin-top: 20px;
|
||||
background: white;
|
||||
}
|
||||
th, td {
|
||||
padding: 15px;
|
||||
text-align: left;
|
||||
border-bottom: 1px solid #e0e0e0;
|
||||
}
|
||||
th {
|
||||
background: #f8f9fa;
|
||||
font-weight: 600;
|
||||
color: #495057;
|
||||
position: sticky;
|
||||
top: 0;
|
||||
}
|
||||
tr:hover { background: #f8f9fa; }
|
||||
|
||||
.status-active {
|
||||
color: #51cf66;
|
||||
font-weight: 600;
|
||||
}
|
||||
.status-inactive {
|
||||
color: #ff6b6b;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.btn {
|
||||
padding: 8px 16px;
|
||||
border: none;
|
||||
border-radius: 6px;
|
||||
cursor: pointer;
|
||||
font-size: 14px;
|
||||
transition: all 0.3s;
|
||||
font-weight: 500;
|
||||
}
|
||||
.btn-primary {
|
||||
background: #667eea;
|
||||
color: white;
|
||||
}
|
||||
.btn-primary:hover {
|
||||
background: #5568d3;
|
||||
transform: translateY(-2px);
|
||||
}
|
||||
.btn-danger {
|
||||
background: #ff6b6b;
|
||||
color: white;
|
||||
}
|
||||
.btn-danger:hover {
|
||||
background: #ee5a52;
|
||||
transform: translateY(-2px);
|
||||
}
|
||||
|
||||
.loading {
|
||||
text-align: center;
|
||||
padding: 40px;
|
||||
color: #666;
|
||||
}
|
||||
|
||||
.empty-state {
|
||||
text-align: center;
|
||||
padding: 60px;
|
||||
color: #999;
|
||||
}
|
||||
.empty-state-icon {
|
||||
font-size: 64px;
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<a href="{{ root_path }}/" class="back-link">← Back to Dashboard</a>
|
||||
|
||||
<div class="header">
|
||||
<h1>👥 User Management</h1>
|
||||
{% if user %}
|
||||
<div class="user-info">
|
||||
<span>{{ user.name or user.username }}</span>
|
||||
{% if user.roles %}
|
||||
{% for role in user.roles %}
|
||||
<span class="role-badge role-{{ role }}">{{ role }}</span>
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
|
||||
<div id="alertContainer"></div>
|
||||
|
||||
<div class="stats" id="statsContainer">
|
||||
<div class="stat-card">
|
||||
<div class="stat-number" id="totalUsers">-</div>
|
||||
<div class="stat-label">Total Users</div>
|
||||
</div>
|
||||
<div class="stat-card">
|
||||
<div class="stat-number" id="activeUsers">-</div>
|
||||
<div class="stat-label">Active Users</div>
|
||||
</div>
|
||||
<div class="stat-card">
|
||||
<div class="stat-number" id="adminUsers">-</div>
|
||||
<div class="stat-label">Admins</div>
|
||||
</div>
|
||||
<div class="stat-card">
|
||||
<div class="stat-number" id="operationUsers">-</div>
|
||||
<div class="stat-label">Operations</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="tableContainer">
|
||||
<div class="loading">Loading users...</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
const rootPath = "{{ root_path }}";
|
||||
|
||||
async function loadUsers() {
|
||||
try {
|
||||
const response = await fetch(`${rootPath}/admin/users/`);
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
|
||||
}
|
||||
|
||||
const users = await response.json();
|
||||
|
||||
// Update stats
|
||||
document.getElementById('totalUsers').textContent = users.length;
|
||||
document.getElementById('activeUsers').textContent = users.filter(u => u.is_active).length;
|
||||
document.getElementById('adminUsers').textContent = users.filter(u =>
|
||||
u.roles.some(r => r.name === 'admin')
|
||||
).length;
|
||||
document.getElementById('operationUsers').textContent = users.filter(u =>
|
||||
u.roles.some(r => r.name === 'operation')
|
||||
).length;
|
||||
|
||||
// Render table
|
||||
const tableContainer = document.getElementById('tableContainer');
|
||||
|
||||
if (users.length === 0) {
|
||||
tableContainer.innerHTML = `
|
||||
<div class="empty-state">
|
||||
<div class="empty-state-icon">📭</div>
|
||||
<h3>No users found</h3>
|
||||
<p>Users will appear here when they log in for the first time.</p>
|
||||
</div>
|
||||
`;
|
||||
return;
|
||||
}
|
||||
|
||||
tableContainer.innerHTML = `
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Username</th>
|
||||
<th>Email</th>
|
||||
<th>Full Name</th>
|
||||
<th>Roles</th>
|
||||
<th>Status</th>
|
||||
<th>Last Login</th>
|
||||
<th>Actions</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
${users.map(user => `
|
||||
<tr>
|
||||
<td><strong>${escapeHtml(user.username)}</strong></td>
|
||||
<td>${user.email ? escapeHtml(user.email) : '-'}</td>
|
||||
<td>${user.full_name ? escapeHtml(user.full_name) : '-'}</td>
|
||||
<td>
|
||||
${user.roles.map(role =>
|
||||
`<span class="role-badge role-${role.name}">${role.name}</span>`
|
||||
).join(' ') || '<span style="color: #999;">No roles</span>'}
|
||||
</td>
|
||||
<td class="${user.is_active ? 'status-active' : 'status-inactive'}">
|
||||
${user.is_active ? '✅ Active' : '❌ Inactive'}
|
||||
</td>
|
||||
<td>${user.last_login ? formatDate(user.last_login) : 'Never'}</td>
|
||||
<td>
|
||||
<button class="btn btn-primary" onclick="editUser(${user.id})">Edit</button>
|
||||
<button class="btn btn-danger" onclick="deleteUser(${user.id}, '${escapeHtml(user.username)}')">Delete</button>
|
||||
</td>
|
||||
</tr>
|
||||
`).join('')}
|
||||
</tbody>
|
||||
</table>
|
||||
`;
|
||||
} catch (error) {
|
||||
console.error('Error loading users:', error);
|
||||
document.getElementById('tableContainer').innerHTML = `
|
||||
<div class="empty-state">
|
||||
<div class="empty-state-icon">⚠️</div>
|
||||
<h3>Error loading users</h3>
|
||||
<p>${escapeHtml(error.message)}</p>
|
||||
</div>
|
||||
`;
|
||||
showAlert('Failed to load users: ' + error.message, 'error');
|
||||
}
|
||||
}
|
||||
|
||||
function showAlert(message, type) {
|
||||
const alertContainer = document.getElementById('alertContainer');
|
||||
alertContainer.innerHTML = `
|
||||
<div class="alert alert-${type}">
|
||||
${escapeHtml(message)}
|
||||
</div>
|
||||
`;
|
||||
setTimeout(() => alertContainer.innerHTML = '', 5000);
|
||||
}
|
||||
|
||||
function editUser(userId) {
|
||||
// TODO: Implement edit modal
|
||||
showAlert('Edit functionality coming soon!', 'success');
|
||||
}
|
||||
|
||||
async function deleteUser(userId, username) {
|
||||
if (!confirm(`Are you sure you want to delete user "${username}"?`)) return;
|
||||
|
||||
try {
|
||||
const response = await fetch(`${rootPath}/admin/users/${userId}`, {
|
||||
method: 'DELETE'
|
||||
});
|
||||
|
||||
if (response.ok) {
|
||||
showAlert(`User "${username}" deleted successfully`, 'success');
|
||||
loadUsers();
|
||||
} else {
|
||||
const error = await response.json();
|
||||
showAlert('Failed to delete user: ' + error.detail, 'error');
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Error deleting user:', error);
|
||||
showAlert('Failed to delete user: ' + error.message, 'error');
|
||||
}
|
||||
}
|
||||
|
||||
function formatDate(dateString) {
|
||||
const date = new Date(dateString);
|
||||
return date.toLocaleString('en-US', {
|
||||
year: 'numeric',
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
hour: '2-digit',
|
||||
minute: '2-digit'
|
||||
});
|
||||
}
|
||||
|
||||
function escapeHtml(text) {
|
||||
const div = document.createElement('div');
|
||||
div.textContent = text;
|
||||
return div.innerHTML;
|
||||
}
|
||||
|
||||
// Load users on page load
|
||||
loadUsers();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
635
03-apiservice/app/templates/data_management_finance.html
Normal file
635
03-apiservice/app/templates/data_management_finance.html
Normal file
@@ -0,0 +1,635 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Finance Excel Upload - Sriphat Data Platform</title>
|
||||
<style>
|
||||
* {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
body {
|
||||
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
|
||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
||||
min-height: 100vh;
|
||||
padding: 20px;
|
||||
}
|
||||
|
||||
.container {
|
||||
max-width: 1000px;
|
||||
margin: 0 auto;
|
||||
}
|
||||
|
||||
.header {
|
||||
background: white;
|
||||
border-radius: 12px;
|
||||
padding: 25px;
|
||||
margin-bottom: 20px;
|
||||
box-shadow: 0 4px 6px rgba(0,0,0,0.1);
|
||||
position: relative;
|
||||
}
|
||||
|
||||
.header h1 {
|
||||
color: #333;
|
||||
margin-bottom: 10px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
}
|
||||
|
||||
.header p {
|
||||
color: #666;
|
||||
}
|
||||
|
||||
.user-info {
|
||||
position: absolute;
|
||||
top: 25px;
|
||||
right: 25px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 15px;
|
||||
background: #f8f9fa;
|
||||
padding: 8px 16px;
|
||||
border-radius: 20px;
|
||||
border: 1px solid #dee2e6;
|
||||
}
|
||||
|
||||
.user-info .username {
|
||||
font-size: 0.9rem;
|
||||
color: #495057;
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.logout-btn {
|
||||
background: #667eea;
|
||||
color: white;
|
||||
padding: 5px 12px;
|
||||
border-radius: 12px;
|
||||
text-decoration: none;
|
||||
font-size: 0.85rem;
|
||||
transition: background 0.3s ease;
|
||||
}
|
||||
|
||||
.logout-btn:hover {
|
||||
background: #5568d3;
|
||||
}
|
||||
|
||||
.role-badge {
|
||||
padding: 4px 12px;
|
||||
border-radius: 12px;
|
||||
font-size: 11px;
|
||||
font-weight: bold;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
}
|
||||
.role-admin {
|
||||
background: linear-gradient(135deg, #ff6b6b 0%, #ee5a52 100%);
|
||||
color: white;
|
||||
box-shadow: 0 2px 4px rgba(255,107,107,0.3);
|
||||
}
|
||||
.role-operation {
|
||||
background: linear-gradient(135deg, #4ecdc4 0%, #44a08d 100%);
|
||||
color: white;
|
||||
box-shadow: 0 2px 4px rgba(78,205,196,0.3);
|
||||
}
|
||||
|
||||
.back-link {
|
||||
display: inline-block;
|
||||
color: #667eea;
|
||||
text-decoration: none;
|
||||
margin-bottom: 15px;
|
||||
font-size: 0.9rem;
|
||||
}
|
||||
|
||||
.back-link:hover {
|
||||
text-decoration: underline;
|
||||
}
|
||||
|
||||
.upload-section {
|
||||
background: white;
|
||||
border-radius: 12px;
|
||||
padding: 30px;
|
||||
margin-bottom: 20px;
|
||||
box-shadow: 0 4px 6px rgba(0,0,0,0.1);
|
||||
}
|
||||
|
||||
.upload-section h2 {
|
||||
color: #333;
|
||||
margin-bottom: 20px;
|
||||
font-size: 1.5rem;
|
||||
}
|
||||
|
||||
.upload-form {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 20px;
|
||||
}
|
||||
|
||||
.form-group {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.form-group label {
|
||||
color: #333;
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.file-input-wrapper {
|
||||
position: relative;
|
||||
display: inline-block;
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.file-input {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.file-input-label {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
padding: 40px;
|
||||
border: 2px dashed #667eea;
|
||||
border-radius: 8px;
|
||||
background: #f8f9fa;
|
||||
cursor: pointer;
|
||||
transition: all 0.3s ease;
|
||||
}
|
||||
|
||||
.file-input-label:hover {
|
||||
background: #e9ecef;
|
||||
border-color: #764ba2;
|
||||
}
|
||||
|
||||
.file-input-label.has-file {
|
||||
border-color: #28a745;
|
||||
background: #d4edda;
|
||||
}
|
||||
|
||||
.file-info {
|
||||
display: none;
|
||||
margin-top: 10px;
|
||||
padding: 10px;
|
||||
background: #e9ecef;
|
||||
border-radius: 6px;
|
||||
font-size: 0.9rem;
|
||||
}
|
||||
|
||||
.file-info.show {
|
||||
display: block;
|
||||
}
|
||||
|
||||
.btn {
|
||||
padding: 12px 30px;
|
||||
border: none;
|
||||
border-radius: 6px;
|
||||
font-size: 1rem;
|
||||
cursor: pointer;
|
||||
transition: all 0.3s ease;
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.btn-primary {
|
||||
background: #667eea;
|
||||
color: white;
|
||||
}
|
||||
|
||||
.btn-primary:hover {
|
||||
background: #5568d3;
|
||||
}
|
||||
|
||||
.btn-primary:disabled {
|
||||
background: #ccc;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.alert {
|
||||
padding: 15px;
|
||||
border-radius: 6px;
|
||||
margin-bottom: 20px;
|
||||
display: none;
|
||||
}
|
||||
|
||||
.alert.show {
|
||||
display: block;
|
||||
}
|
||||
|
||||
.alert-success {
|
||||
background: #d4edda;
|
||||
color: #155724;
|
||||
border: 1px solid #c3e6cb;
|
||||
}
|
||||
|
||||
.alert-error {
|
||||
background: #f8d7da;
|
||||
color: #721c24;
|
||||
border: 1px solid #f5c6cb;
|
||||
}
|
||||
|
||||
.alert-info {
|
||||
background: #d1ecf1;
|
||||
color: #0c5460;
|
||||
border: 1px solid #bee5eb;
|
||||
}
|
||||
|
||||
.uploads-list {
|
||||
background: white;
|
||||
border-radius: 12px;
|
||||
padding: 30px;
|
||||
box-shadow: 0 4px 6px rgba(0,0,0,0.1);
|
||||
}
|
||||
|
||||
.uploads-list h2 {
|
||||
color: #333;
|
||||
margin-bottom: 20px;
|
||||
font-size: 1.5rem;
|
||||
}
|
||||
|
||||
.upload-item {
|
||||
border: 1px solid #dee2e6;
|
||||
border-radius: 8px;
|
||||
margin-bottom: 15px;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.upload-header {
|
||||
padding: 15px;
|
||||
background: #f8f9fa;
|
||||
cursor: pointer;
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
transition: background 0.2s ease;
|
||||
}
|
||||
|
||||
.upload-header:hover {
|
||||
background: #e9ecef;
|
||||
}
|
||||
|
||||
.upload-info {
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
.upload-filename {
|
||||
font-weight: 500;
|
||||
color: #333;
|
||||
margin-bottom: 5px;
|
||||
}
|
||||
|
||||
.upload-meta {
|
||||
font-size: 0.85rem;
|
||||
color: #666;
|
||||
}
|
||||
|
||||
.upload-status {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
}
|
||||
|
||||
.status-badge {
|
||||
padding: 5px 12px;
|
||||
border-radius: 20px;
|
||||
font-size: 0.85rem;
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.status-pending {
|
||||
background: #fff3cd;
|
||||
color: #856404;
|
||||
}
|
||||
|
||||
.status-processing {
|
||||
background: #d1ecf1;
|
||||
color: #0c5460;
|
||||
}
|
||||
|
||||
.status-success {
|
||||
background: #d4edda;
|
||||
color: #155724;
|
||||
}
|
||||
|
||||
.status-error {
|
||||
background: #f8d7da;
|
||||
color: #721c24;
|
||||
}
|
||||
|
||||
.expand-icon {
|
||||
transition: transform 0.3s ease;
|
||||
}
|
||||
|
||||
.expand-icon.expanded {
|
||||
transform: rotate(180deg);
|
||||
}
|
||||
|
||||
.upload-details {
|
||||
padding: 15px;
|
||||
background: white;
|
||||
border-top: 1px solid #dee2e6;
|
||||
display: none;
|
||||
}
|
||||
|
||||
.upload-details.show {
|
||||
display: block;
|
||||
}
|
||||
|
||||
.log-container {
|
||||
background: #f8f9fa;
|
||||
border: 1px solid #dee2e6;
|
||||
border-radius: 6px;
|
||||
padding: 15px;
|
||||
font-family: 'Courier New', monospace;
|
||||
font-size: 0.85rem;
|
||||
max-height: 300px;
|
||||
overflow-y: auto;
|
||||
white-space: pre-wrap;
|
||||
word-wrap: break-word;
|
||||
}
|
||||
|
||||
.log-error {
|
||||
color: #dc3545;
|
||||
}
|
||||
|
||||
.log-info {
|
||||
color: #17a2b8;
|
||||
}
|
||||
|
||||
.empty-state {
|
||||
text-align: center;
|
||||
padding: 40px;
|
||||
color: #666;
|
||||
}
|
||||
|
||||
.empty-state .icon {
|
||||
font-size: 3rem;
|
||||
margin-bottom: 15px;
|
||||
opacity: 0.5;
|
||||
}
|
||||
|
||||
.spinner {
|
||||
display: inline-block;
|
||||
width: 20px;
|
||||
height: 20px;
|
||||
border: 3px solid rgba(0,0,0,0.1);
|
||||
border-radius: 50%;
|
||||
border-top-color: #667eea;
|
||||
animation: spin 1s ease-in-out infinite;
|
||||
}
|
||||
|
||||
@keyframes spin {
|
||||
to { transform: rotate(360deg); }
|
||||
}
|
||||
|
||||
@media (max-width: 768px) {
|
||||
.container {
|
||||
padding: 10px;
|
||||
}
|
||||
|
||||
.upload-section, .uploads-list {
|
||||
padding: 20px;
|
||||
}
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<a href="{{ root_path }}/" class="back-link">← Back to Dashboard</a>
|
||||
|
||||
<div class="header">
|
||||
<h1>💰 Finance Excel Upload</h1>
|
||||
<p>Upload Excel files for financial data processing</p>
|
||||
|
||||
{% if user %}
|
||||
<div class="user-info">
|
||||
<span class="username">👤 {{ user.name or user.username }}</span>
|
||||
{% if user.roles %}
|
||||
{% for role in user.roles %}
|
||||
<span class="role-badge role-{{ role }}">{{ role }}</span>
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
<a href="{{ root_path }}/auth/logout" class="logout-btn">Logout</a>
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
|
||||
<div id="alertContainer"></div>
|
||||
|
||||
<div class="upload-section">
|
||||
<h2>📤 Upload File</h2>
|
||||
<form id="uploadForm" class="upload-form" enctype="multipart/form-data">
|
||||
<div class="form-group">
|
||||
<label for="file">Select Excel File (.xlsx, .xls)</label>
|
||||
<div class="file-input-wrapper">
|
||||
<input type="file" id="file" name="file" class="file-input" accept=".xlsx,.xls" required>
|
||||
<label for="file" id="fileLabel" class="file-input-label">
|
||||
<span>📁 Click to select file or drag and drop</span>
|
||||
</label>
|
||||
</div>
|
||||
<div id="fileInfo" class="file-info"></div>
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="description">Description (Optional)</label>
|
||||
<textarea id="description" name="description" rows="3" style="padding: 10px; border: 1px solid #dee2e6; border-radius: 6px; font-family: inherit;"></textarea>
|
||||
</div>
|
||||
|
||||
<button type="submit" class="btn btn-primary" id="submitBtn">
|
||||
Upload and Process
|
||||
</button>
|
||||
</form>
|
||||
</div>
|
||||
|
||||
<div class="uploads-list">
|
||||
<h2>📋 Upload History</h2>
|
||||
<div id="uploadsList">
|
||||
<div class="empty-state">
|
||||
<div class="icon">📭</div>
|
||||
<p>No uploads yet</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
const rootPath = '{{ root_path }}';
|
||||
const fileInput = document.getElementById('file');
|
||||
const fileLabel = document.getElementById('fileLabel');
|
||||
const fileInfo = document.getElementById('fileInfo');
|
||||
const uploadForm = document.getElementById('uploadForm');
|
||||
const submitBtn = document.getElementById('submitBtn');
|
||||
const uploadsList = document.getElementById('uploadsList');
|
||||
const alertContainer = document.getElementById('alertContainer');
|
||||
|
||||
// File input handling
|
||||
fileInput.addEventListener('change', function(e) {
|
||||
const file = e.target.files[0];
|
||||
if (file) {
|
||||
fileLabel.classList.add('has-file');
|
||||
fileLabel.innerHTML = `<span>✅ ${file.name}</span>`;
|
||||
fileInfo.classList.add('show');
|
||||
fileInfo.textContent = `File: ${file.name} (${formatFileSize(file.size)})`;
|
||||
}
|
||||
});
|
||||
|
||||
// Drag and drop
|
||||
fileLabel.addEventListener('dragover', (e) => {
|
||||
e.preventDefault();
|
||||
fileLabel.style.borderColor = '#764ba2';
|
||||
});
|
||||
|
||||
fileLabel.addEventListener('dragleave', () => {
|
||||
fileLabel.style.borderColor = '#667eea';
|
||||
});
|
||||
|
||||
fileLabel.addEventListener('drop', (e) => {
|
||||
e.preventDefault();
|
||||
fileLabel.style.borderColor = '#667eea';
|
||||
const files = e.dataTransfer.files;
|
||||
if (files.length > 0) {
|
||||
fileInput.files = files;
|
||||
fileInput.dispatchEvent(new Event('change'));
|
||||
}
|
||||
});
|
||||
|
||||
// Form submission
|
||||
uploadForm.addEventListener('submit', async function(e) {
|
||||
e.preventDefault();
|
||||
|
||||
const formData = new FormData(uploadForm);
|
||||
|
||||
submitBtn.disabled = true;
|
||||
submitBtn.innerHTML = '<span class="spinner"></span> Uploading...';
|
||||
|
||||
try {
|
||||
const response = await fetch(`${rootPath}/data-management/finance/upload`, {
|
||||
method: 'POST',
|
||||
body: formData
|
||||
});
|
||||
|
||||
const result = await response.json();
|
||||
|
||||
if (response.ok) {
|
||||
showAlert('success', result.message || 'File uploaded successfully!');
|
||||
uploadForm.reset();
|
||||
fileLabel.classList.remove('has-file');
|
||||
fileLabel.innerHTML = '<span>📁 Click to select file or drag and drop</span>';
|
||||
fileInfo.classList.remove('show');
|
||||
|
||||
// Refresh uploads list
|
||||
loadUploads();
|
||||
} else {
|
||||
showAlert('error', result.detail || 'Upload failed');
|
||||
}
|
||||
} catch (error) {
|
||||
showAlert('error', 'Network error: ' + error.message);
|
||||
} finally {
|
||||
submitBtn.disabled = false;
|
||||
submitBtn.textContent = 'Upload and Process';
|
||||
}
|
||||
});
|
||||
|
||||
// Load uploads list
|
||||
async function loadUploads() {
|
||||
try {
|
||||
const response = await fetch(`${rootPath}/data-management/finance/uploads`);
|
||||
const uploads = await response.json();
|
||||
|
||||
if (uploads.length === 0) {
|
||||
uploadsList.innerHTML = `
|
||||
<div class="empty-state">
|
||||
<div class="icon">📭</div>
|
||||
<p>No uploads yet</p>
|
||||
</div>
|
||||
`;
|
||||
return;
|
||||
}
|
||||
|
||||
uploadsList.innerHTML = uploads.map(upload => `
|
||||
<div class="upload-item">
|
||||
<div class="upload-header" onclick="toggleDetails('${upload.id}')">
|
||||
<div class="upload-info">
|
||||
<div class="upload-filename">${upload.filename}</div>
|
||||
<div class="upload-meta">
|
||||
Uploaded: ${formatDate(upload.uploaded_at)}
|
||||
${upload.description ? `• ${upload.description}` : ''}
|
||||
</div>
|
||||
</div>
|
||||
<div class="upload-status">
|
||||
<span class="status-badge status-${upload.status}">${upload.status.toUpperCase()}</span>
|
||||
<span class="expand-icon" id="icon-${upload.id}">▼</span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="upload-details" id="details-${upload.id}">
|
||||
${upload.job_id ? `<p><strong>Job ID:</strong> ${upload.job_id}</p>` : ''}
|
||||
${upload.logs ? `
|
||||
<h4>Logs:</h4>
|
||||
<div class="log-container ${upload.status === 'error' ? 'log-error' : 'log-info'}">
|
||||
${upload.logs}
|
||||
</div>
|
||||
` : '<p>No logs available</p>'}
|
||||
</div>
|
||||
</div>
|
||||
`).join('');
|
||||
} catch (error) {
|
||||
console.error('Failed to load uploads:', error);
|
||||
}
|
||||
}
|
||||
|
||||
function toggleDetails(id) {
|
||||
const details = document.getElementById(`details-${id}`);
|
||||
const icon = document.getElementById(`icon-${id}`);
|
||||
|
||||
if (details.classList.contains('show')) {
|
||||
details.classList.remove('show');
|
||||
icon.classList.remove('expanded');
|
||||
} else {
|
||||
details.classList.add('show');
|
||||
icon.classList.add('expanded');
|
||||
}
|
||||
}
|
||||
|
||||
function showAlert(type, message) {
|
||||
const alertClass = type === 'success' ? 'alert-success' : type === 'error' ? 'alert-error' : 'alert-info';
|
||||
const alert = document.createElement('div');
|
||||
alert.className = `alert ${alertClass} show`;
|
||||
alert.textContent = message;
|
||||
alertContainer.innerHTML = '';
|
||||
alertContainer.appendChild(alert);
|
||||
|
||||
setTimeout(() => {
|
||||
alert.classList.remove('show');
|
||||
}, 5000);
|
||||
}
|
||||
|
||||
function formatFileSize(bytes) {
|
||||
if (bytes === 0) return '0 Bytes';
|
||||
const k = 1024;
|
||||
const sizes = ['Bytes', 'KB', 'MB', 'GB'];
|
||||
const i = Math.floor(Math.log(bytes) / Math.log(k));
|
||||
return Math.round(bytes / Math.pow(k, i) * 100) / 100 + ' ' + sizes[i];
|
||||
}
|
||||
|
||||
function formatDate(dateString) {
|
||||
const date = new Date(dateString);
|
||||
return date.toLocaleString('en-US', {
|
||||
year: 'numeric',
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
hour: '2-digit',
|
||||
minute: '2-digit'
|
||||
});
|
||||
}
|
||||
|
||||
// Load uploads on page load
|
||||
loadUploads();
|
||||
|
||||
// Refresh uploads every 10 seconds
|
||||
setInterval(loadUploads, 10000);
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
296
03-apiservice/app/templates/index.html
Normal file
296
03-apiservice/app/templates/index.html
Normal file
@@ -0,0 +1,296 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Sriphat Data Platform</title>
|
||||
<style>
|
||||
* {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
body {
|
||||
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
|
||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
||||
min-height: 100vh;
|
||||
padding: 20px;
|
||||
}
|
||||
|
||||
.container {
|
||||
max-width: 1200px;
|
||||
margin: 0 auto;
|
||||
}
|
||||
|
||||
.header {
|
||||
text-align: center;
|
||||
color: white;
|
||||
margin-bottom: 40px;
|
||||
padding: 20px;
|
||||
position: relative;
|
||||
}
|
||||
|
||||
.header h1 {
|
||||
font-size: 2.5rem;
|
||||
margin-bottom: 10px;
|
||||
text-shadow: 2px 2px 4px rgba(0,0,0,0.3);
|
||||
}
|
||||
|
||||
.header p {
|
||||
font-size: 1.1rem;
|
||||
opacity: 0.9;
|
||||
}
|
||||
|
||||
.user-info {
|
||||
position: absolute;
|
||||
top: 20px;
|
||||
right: 20px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 15px;
|
||||
background: rgba(255,255,255,0.1);
|
||||
padding: 10px 20px;
|
||||
border-radius: 25px;
|
||||
backdrop-filter: blur(10px);
|
||||
}
|
||||
|
||||
.user-info .username {
|
||||
font-size: 0.95rem;
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.logout-btn {
|
||||
background: rgba(255,255,255,0.2);
|
||||
color: white;
|
||||
padding: 6px 15px;
|
||||
border-radius: 15px;
|
||||
text-decoration: none;
|
||||
font-size: 0.9rem;
|
||||
transition: background 0.3s ease;
|
||||
}
|
||||
|
||||
.logout-btn:hover {
|
||||
background: rgba(255,255,255,0.3);
|
||||
}
|
||||
|
||||
.role-badge {
|
||||
padding: 4px 12px;
|
||||
border-radius: 12px;
|
||||
font-size: 11px;
|
||||
font-weight: bold;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
}
|
||||
.role-admin {
|
||||
background: linear-gradient(135deg, #ff6b6b 0%, #ee5a52 100%);
|
||||
color: white;
|
||||
box-shadow: 0 2px 4px rgba(255,107,107,0.3);
|
||||
}
|
||||
.role-operation {
|
||||
background: linear-gradient(135deg, #4ecdc4 0%, #44a08d 100%);
|
||||
color: white;
|
||||
box-shadow: 0 2px 4px rgba(78,205,196,0.3);
|
||||
}
|
||||
|
||||
.menu-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(280px, 1fr));
|
||||
gap: 20px;
|
||||
margin-bottom: 30px;
|
||||
}
|
||||
|
||||
.menu-card {
|
||||
background: white;
|
||||
border-radius: 12px;
|
||||
padding: 25px;
|
||||
box-shadow: 0 4px 6px rgba(0,0,0,0.1);
|
||||
transition: transform 0.3s ease, box-shadow 0.3s ease;
|
||||
cursor: pointer;
|
||||
text-decoration: none;
|
||||
color: inherit;
|
||||
display: block;
|
||||
}
|
||||
|
||||
.menu-card:hover {
|
||||
transform: translateY(-5px);
|
||||
box-shadow: 0 8px 15px rgba(0,0,0,0.2);
|
||||
}
|
||||
|
||||
.menu-card.has-submenu {
|
||||
cursor: default;
|
||||
}
|
||||
|
||||
.menu-card .icon {
|
||||
font-size: 2.5rem;
|
||||
margin-bottom: 15px;
|
||||
display: block;
|
||||
}
|
||||
|
||||
.menu-card h3 {
|
||||
color: #333;
|
||||
margin-bottom: 10px;
|
||||
font-size: 1.3rem;
|
||||
}
|
||||
|
||||
.menu-card p {
|
||||
color: #666;
|
||||
font-size: 0.9rem;
|
||||
line-height: 1.5;
|
||||
}
|
||||
|
||||
.submenu {
|
||||
margin-top: 15px;
|
||||
padding-top: 15px;
|
||||
border-top: 1px solid #eee;
|
||||
display: none;
|
||||
}
|
||||
|
||||
.submenu.active {
|
||||
display: block;
|
||||
}
|
||||
|
||||
.submenu-item {
|
||||
display: block;
|
||||
padding: 10px 15px;
|
||||
margin: 5px 0;
|
||||
background: #f8f9fa;
|
||||
border-radius: 6px;
|
||||
color: #495057;
|
||||
text-decoration: none;
|
||||
transition: background 0.2s ease;
|
||||
}
|
||||
|
||||
.submenu-item:hover {
|
||||
background: #e9ecef;
|
||||
}
|
||||
|
||||
.submenu-toggle {
|
||||
display: inline-block;
|
||||
margin-left: 10px;
|
||||
font-size: 0.8rem;
|
||||
color: #667eea;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
/* Color themes for different services */
|
||||
.card-supabase { border-left: 4px solid #3ECF8E; }
|
||||
.card-api { border-left: 4px solid #009688; }
|
||||
.card-airflow { border-left: 4px solid #017CEE; }
|
||||
.card-airbyte { border-left: 4px solid #615EFF; }
|
||||
.card-data { border-left: 4px solid #FF6B6B; }
|
||||
.card-dbt { border-left: 4px solid #FF694B; }
|
||||
.card-superset { border-left: 4px solid #20A7C9; }
|
||||
|
||||
@media (max-width: 768px) {
|
||||
.header h1 {
|
||||
font-size: 2rem;
|
||||
}
|
||||
|
||||
.menu-grid {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<div class="header">
|
||||
<h1>🏥 Sriphat Data Platform</h1>
|
||||
<p>Integrated Data Management & Analytics Platform</p>
|
||||
|
||||
{% if user %}
|
||||
<div class="user-info">
|
||||
<span class="username">👤 {{ user.name or user.username }}</span>
|
||||
{% if user.roles %}
|
||||
{% for role in user.roles %}
|
||||
<span class="role-badge role-{{ role }}">{{ role }}</span>
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
<a href="{{ root_path }}/auth/logout" class="logout-btn">Logout</a>
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
|
||||
<div class="menu-grid">
|
||||
<!-- Supabase -->
|
||||
<a href="https://ai.sriphat.com/supabase" class="menu-card card-supabase" target="_blank">
|
||||
<span class="icon">🗄️</span>
|
||||
<h3>Supabase</h3>
|
||||
<p>PostgreSQL database with real-time capabilities and REST API</p>
|
||||
</a>
|
||||
|
||||
<!-- API Docs -->
|
||||
<a href="{{ root_path }}/docs" class="menu-card card-api">
|
||||
<span class="icon">📚</span>
|
||||
<h3>API Documentation</h3>
|
||||
<p>Interactive API documentation and testing interface</p>
|
||||
</a>
|
||||
|
||||
<!-- Airflow -->
|
||||
<a href="https://ai.sriphat.com/airflow" class="menu-card card-airflow" target="_blank">
|
||||
<span class="icon">🔄</span>
|
||||
<h3>Airflow</h3>
|
||||
<p>Workflow orchestration and data pipeline management</p>
|
||||
</a>
|
||||
|
||||
<!-- Airbyte -->
|
||||
<a href="https://ai.sriphat.com/airbyte" class="menu-card card-airbyte" target="_blank">
|
||||
<span class="icon">🔌</span>
|
||||
<h3>Airbyte</h3>
|
||||
<p>Data integration and ETL platform</p>
|
||||
</a>
|
||||
|
||||
<!-- Data Management -->
|
||||
<div class="menu-card card-data has-submenu" onclick="toggleSubmenu(this)">
|
||||
<span class="icon">📊</span>
|
||||
<h3>Data Management <span class="submenu-toggle">▼</span></h3>
|
||||
<p>Upload and manage data files</p>
|
||||
<div class="submenu">
|
||||
<a href="{{ root_path }}/data-management/finance" class="submenu-item">
|
||||
💰 Finance Excel Upload
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- DBT -->
|
||||
<a href="https://ai.sriphat.com/dbt" class="menu-card card-dbt" target="_blank">
|
||||
<span class="icon">🔧</span>
|
||||
<h3>DBT</h3>
|
||||
<p>Data transformation and modeling</p>
|
||||
</a>
|
||||
|
||||
<!-- Superset -->
|
||||
<a href="https://ai.sriphat.com/superset" class="menu-card card-superset" target="_blank">
|
||||
<span class="icon">📈</span>
|
||||
<h3>Superset</h3>
|
||||
<p>Business intelligence and data visualization</p>
|
||||
</a>
|
||||
|
||||
<!-- User Management (Admin only) -->
|
||||
{% if user and user.roles and 'admin' in user.roles %}
|
||||
<a href="{{ root_path }}/admin/users" class="menu-card card-admin">
|
||||
<span class="icon">👥</span>
|
||||
<h3>User Management</h3>
|
||||
<p>Manage users and roles (Admin only)</p>
|
||||
</a>
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
function toggleSubmenu(card) {
|
||||
const submenu = card.querySelector('.submenu');
|
||||
const toggle = card.querySelector('.submenu-toggle');
|
||||
|
||||
if (submenu.classList.contains('active')) {
|
||||
submenu.classList.remove('active');
|
||||
toggle.textContent = '▼';
|
||||
} else {
|
||||
submenu.classList.add('active');
|
||||
toggle.textContent = '▲';
|
||||
}
|
||||
}
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -48,7 +48,7 @@ async def upsert_to_supabase(
|
||||
headers["Prefer"] += f",on_conflict={on_conflict}"
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||
async with httpx.AsyncClient(timeout=30.0, verify=False) as client:
|
||||
response = await client.post(url, json=data, headers=headers)
|
||||
response.raise_for_status()
|
||||
return {
|
||||
@@ -102,7 +102,7 @@ def upsert_to_supabase_sync(
|
||||
headers["Prefer"] += f",on_conflict={on_conflict}"
|
||||
|
||||
try:
|
||||
with httpx.Client(timeout=30.0) as client:
|
||||
with httpx.Client(timeout=30.0, verify=False) as client:
|
||||
response = client.post(url, json=data, headers=headers)
|
||||
response.raise_for_status()
|
||||
return {
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
x-common-configs: &common-config
|
||||
extra_hosts:
|
||||
- "dev.sriphat.com:192.168.100.9"
|
||||
pull_policy: ${DOCKER_PULL_POLICY:-missing}
|
||||
|
||||
services:
|
||||
apiservice:
|
||||
build: .
|
||||
#build: .
|
||||
image: 03-apiservice-apiservice:latest
|
||||
container_name: apiservice
|
||||
env_file:
|
||||
- .env
|
||||
@@ -17,6 +23,13 @@ services:
|
||||
- ADMIN_SECRET_KEY=${ADMIN_SECRET_KEY}
|
||||
- ADMIN_USERNAME=${ADMIN_USERNAME}
|
||||
- ADMIN_PASSWORD=${ADMIN_PASSWORD}
|
||||
- API_KEY_ENC_SECRET=${ADMIN_SECRET_KEY}
|
||||
- DEBUG_AUTH=${DEBUG_AUTH:-false}
|
||||
- KEYCLOAK_SERVER_URL=${KEYCLOAK_SERVER_URL}
|
||||
- KEYCLOAK_REALM=${KEYCLOAK_REALM}
|
||||
- KEYCLOAK_CLIENT_ID=${API_KEYCLOAK_CLIENT_ID}
|
||||
- KEYCLOAK_CLIENT_SECRET=${API_KEYCLOAK_CLIENT_SECRET}
|
||||
- KEYCLOAK_REDIRECT_URI=${API_KEYCLOAK_REDIRECT_URI}
|
||||
- LOG_LEVEL=debug
|
||||
ports:
|
||||
- "8040:8040"
|
||||
@@ -25,6 +38,7 @@ services:
|
||||
volumes:
|
||||
- ./app:/app/app
|
||||
- .env:/app/.env
|
||||
- ./data/uploads:/data/uploads
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:8040/apiservice/docs"]
|
||||
@@ -32,6 +46,9 @@ services:
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 40s
|
||||
# extra_hosts:
|
||||
# - "dev.sriphat.com:192.168.100.9"
|
||||
<<: *common-config
|
||||
|
||||
networks:
|
||||
shared_data_network:
|
||||
|
||||
@@ -14,4 +14,7 @@ httpx==0.28.1
|
||||
WTForms
|
||||
#==3.2.1
|
||||
cryptography==42.0.5
|
||||
python-keycloak==3.9.0
|
||||
Authlib==1.3.0
|
||||
python-jose[cryptography]==3.3.0
|
||||
|
||||
|
||||
Reference in New Issue
Block a user