Files
jigoong a587be08bd feat: MinIO integration — bucket finance, API service upload, Nginx routing
- 01-infra/nginx-configs: add MinIO /minio/ and /minio-console/ location blocks
  (port 9000 S3 API, port 9001 Console UI, path stripping via rewrite)
- 03-apiservice: integrate MinIO minio-python SDK for file upload
  - requirements.txt: add minio==7.2.11
  - app/core/config.py: add MINIO_ENDPOINT, ACCESS_KEY, SECRET_KEY, BUCKET_FINANCE, USE_SSL
  - app/services/minio_client.py: new — upload_file(), get_presigned_url(), delete_file()
  - app/routes/pages.py: replace local /data/uploads/ write with MinIO upload to finance bucket
  - docker-compose.yml: pass MinIO env vars to container
  - .env.example: document MinIO vars
- 07-minio/.env.example: add MINIO_SVC_ACCESS_KEY/SECRET_KEY section
- 07-minio/README.md: add Python minio SDK and Airflow DAG usage guide
- CLAUDE.md: project context (servers, SSH, paths, service distribution)
- document-obsidiant/: initial Obsidian docs for all services
2026-05-20 17:42:39 +07:00

265 lines
6.0 KiB
Markdown

---
tags:
- project/sriphat
- minio
- storage
- s3
created: 2026-05-07
status: active
folder: 07-minio
---
# MinIO Object Storage (07-minio)
> **Docker Compose:** `07-minio/docker-compose.yml`
> **Env File:** `07-minio/.env`
## Overview
MinIO เป็น S3-compatible object storage สำหรับ:
- เก็บ raw data files (CSV, JSON, Parquet)
- เก็บ ML/AI models และ training data
- เก็บ backups และ reports
- Keycloak SSO integration
---
## Container
| รายการ | ค่า |
|--------|-----|
| **Container** | `minio` |
| **Image** | `minio/minio:latest` |
| **API Port** | `9000:9000` |
| **Console Port** | `9001:9001` |
| **Console URL** | `https://ai.sriphat.com/minio-console` |
| **API URL** | `https://ai.sriphat.com/minio` |
| **Direct (Dev)** | `http://192.168.100.9:9001` (console) |
| **Region** | `ap-southeast-1` |
---
## Use Cases
| Use Case | ตัวอย่าง |
|----------|---------|
| **Data Lake** | Raw CSV, JSON, Parquet จาก Airbyte |
| **ML/AI Workflows** | Model files, training datasets, experiment artifacts |
| **Backup Storage** | Database dumps, application backups |
| **Report Files** | Excel, PDF reports จาก Finance |
| **Media Storage** | Images, documents จากระบบ HIS |
| **Application Storage** | File uploads จาก API Service |
---
## Authentication
### 1. Root Credentials (Default)
```bash
MINIO_ROOT_USER=minioadmin
MINIO_ROOT_PASSWORD=<strong-password>
```
### 2. Keycloak SSO (แนะนำ)
เชื่อมต่อผ่าน OpenID Connect:
```bash
MINIO_IDENTITY_OPENID_CONFIG_URL=https://ai.sriphat.com/keycloak/realms/sriphat/.well-known/openid-configuration
MINIO_IDENTITY_OPENID_CLIENT_ID=minio-client
MINIO_IDENTITY_OPENID_CLIENT_SECRET=<secret>
MINIO_IDENTITY_OPENID_CLAIM_NAME=policy
MINIO_IDENTITY_OPENID_SCOPES=openid,profile,email
MINIO_IDENTITY_OPENID_REDIRECT_URI=https://ai.sriphat.com/minio-console/oauth_callback
```
**Policy Mapping:** User ใน Keycloak ต้องมี attribute `policy` ที่ map กับ MinIO policy
---
## Environment Variables
```bash
# Credentials
MINIO_ROOT_USER=minioadmin
MINIO_ROOT_PASSWORD=<secret>
# URLs
MINIO_SERVER_URL=https://ai.sriphat.com/minio
MINIO_BROWSER_REDIRECT_URL=https://ai.sriphat.com/minio-console
# Region
MINIO_REGION=ap-southeast-1
# Keycloak SSO
MINIO_IDENTITY_OPENID_CONFIG_URL=<keycloak-oidc-url>
MINIO_IDENTITY_OPENID_CLIENT_ID=<client-id>
MINIO_IDENTITY_OPENID_CLIENT_SECRET=<secret>
MINIO_IDENTITY_OPENID_CLAIM_NAME=policy
MINIO_IDENTITY_OPENID_SCOPES=openid,profile,email
MINIO_IDENTITY_OPENID_REDIRECT_URI=<redirect-uri>
TZ=Asia/Bangkok
```
---
## Volume Mounts
```
07-minio/
├── data/ → /data (object storage data)
└── certs/ → /root/.minio/certs:ro (SSL certificates)
```
---
## การใช้งาน MinIO Client (mc)
```bash
# Install
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc && sudo mv mc /usr/local/bin/
# Config alias
mc alias set sriphat https://ai.sriphat.com/minio minioadmin <password>
# List buckets
mc ls sriphat
# Create bucket
mc mb sriphat/raw-data
mc mb sriphat/ml-models
mc mb sriphat/backups
mc mb sriphat/reports
# Upload
mc cp data.csv sriphat/raw-data/
mc cp -r ./models/ sriphat/ml-models/
# Set bucket policy
mc anonymous set none sriphat/raw-data # private
mc anonymous set download sriphat/public # public read
```
---
## Python SDK (boto3)
```python
import boto3
from botocore.client import Config
s3 = boto3.client(
's3',
endpoint_url='https://ai.sriphat.com/minio',
aws_access_key_id='minioadmin',
aws_secret_access_key='<password>',
config=Config(signature_version='s3v4'),
region_name='ap-southeast-1'
)
# Upload file
s3.upload_file('data.csv', 'raw-data', 'data.csv')
# Download file
s3.download_file('raw-data', 'data.csv', 'local-data.csv')
# List objects
for obj in s3.list_objects_v2(Bucket='raw-data').get('Contents', []):
print(obj['Key'])
```
---
## Recommended Bucket Structure
```
sriphat/
├── raw-data/ # ข้อมูลดิบจาก Airbyte / HIS
│ ├── his/
│ ├── oracle-lab/
│ └── finance-excel/
├── processed-data/ # ข้อมูลที่ transform แล้ว
├── ml-models/ # ML/AI model files
│ ├── waiting-time/
│ └── patient-flow/
├── reports/ # Excel, PDF reports
├── backups/ # Database backups
│ └── postgres/
└── uploads/ # User uploads จาก API Service
```
---
## Security
```bash
# สร้าง read-only policy
cat > readonly-policy.json << 'EOF'
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:ListBucket"],
"Resource": ["arn:aws:s3:::*"]
}
]
}
EOF
mc admin policy create sriphat readonly-policy readonly-policy.json
# Assign policy ให้ user
mc admin policy attach sriphat readonly-policy --user=analyst-user
```
---
## Health Check
```bash
# ตรวจสอบสถานะ
curl -f http://localhost:9000/minio/health/live
docker exec minio curl -f http://localhost:9000/minio/health/live
```
---
## Backup Strategy
```bash
# Backup data directory
tar -czf minio-backup-$(date +%Y%m%d).tar.gz 07-minio/data/
# Sync to remote
rsync -avz 07-minio/data/ backup-server:/backups/minio/
# Restore
docker compose down
tar -xzf minio-backup-20260501.tar.gz
docker compose up -d
```
---
## Keycloak Setup (สำหรับ SSO)
ดูรายละเอียดที่ `07-minio/KEYCLOAK_INTEGRATION.md`
1. สร้าง Client `minio-client` ใน Keycloak Realm `sriphat`
2. ตั้งค่า Valid Redirect URIs: `https://ai.sriphat.com/minio-console/oauth_callback`
3. สร้าง Client Scope `minio-policy`
4. เพิ่ม User Attribute Mapper `policy`
5. กำหนด `policy` attribute ให้กับ users ตาม MinIO policies
---
## Related
- [[00-Project-Overview]]
- [[01-Infrastructure]] (Keycloak SSO)
- [[07-Security-Strategy]]