- 01-infra/nginx-configs: add MinIO /minio/ and /minio-console/ location blocks (port 9000 S3 API, port 9001 Console UI, path stripping via rewrite) - 03-apiservice: integrate MinIO minio-python SDK for file upload - requirements.txt: add minio==7.2.11 - app/core/config.py: add MINIO_ENDPOINT, ACCESS_KEY, SECRET_KEY, BUCKET_FINANCE, USE_SSL - app/services/minio_client.py: new — upload_file(), get_presigned_url(), delete_file() - app/routes/pages.py: replace local /data/uploads/ write with MinIO upload to finance bucket - docker-compose.yml: pass MinIO env vars to container - .env.example: document MinIO vars - 07-minio/.env.example: add MINIO_SVC_ACCESS_KEY/SECRET_KEY section - 07-minio/README.md: add Python minio SDK and Airflow DAG usage guide - CLAUDE.md: project context (servers, SSH, paths, service distribution) - document-obsidiant/: initial Obsidian docs for all services
265 lines
6.0 KiB
Markdown
265 lines
6.0 KiB
Markdown
---
|
|
tags:
|
|
- project/sriphat
|
|
- minio
|
|
- storage
|
|
- s3
|
|
created: 2026-05-07
|
|
status: active
|
|
folder: 07-minio
|
|
---
|
|
|
|
# MinIO Object Storage (07-minio)
|
|
|
|
> **Docker Compose:** `07-minio/docker-compose.yml`
|
|
> **Env File:** `07-minio/.env`
|
|
|
|
## Overview
|
|
|
|
MinIO เป็น S3-compatible object storage สำหรับ:
|
|
- เก็บ raw data files (CSV, JSON, Parquet)
|
|
- เก็บ ML/AI models และ training data
|
|
- เก็บ backups และ reports
|
|
- Keycloak SSO integration
|
|
|
|
---
|
|
|
|
## Container
|
|
|
|
| รายการ | ค่า |
|
|
|--------|-----|
|
|
| **Container** | `minio` |
|
|
| **Image** | `minio/minio:latest` |
|
|
| **API Port** | `9000:9000` |
|
|
| **Console Port** | `9001:9001` |
|
|
| **Console URL** | `https://ai.sriphat.com/minio-console` |
|
|
| **API URL** | `https://ai.sriphat.com/minio` |
|
|
| **Direct (Dev)** | `http://192.168.100.9:9001` (console) |
|
|
| **Region** | `ap-southeast-1` |
|
|
|
|
---
|
|
|
|
## Use Cases
|
|
|
|
| Use Case | ตัวอย่าง |
|
|
|----------|---------|
|
|
| **Data Lake** | Raw CSV, JSON, Parquet จาก Airbyte |
|
|
| **ML/AI Workflows** | Model files, training datasets, experiment artifacts |
|
|
| **Backup Storage** | Database dumps, application backups |
|
|
| **Report Files** | Excel, PDF reports จาก Finance |
|
|
| **Media Storage** | Images, documents จากระบบ HIS |
|
|
| **Application Storage** | File uploads จาก API Service |
|
|
|
|
---
|
|
|
|
## Authentication
|
|
|
|
### 1. Root Credentials (Default)
|
|
|
|
```bash
|
|
MINIO_ROOT_USER=minioadmin
|
|
MINIO_ROOT_PASSWORD=<strong-password>
|
|
```
|
|
|
|
### 2. Keycloak SSO (แนะนำ)
|
|
|
|
เชื่อมต่อผ่าน OpenID Connect:
|
|
|
|
```bash
|
|
MINIO_IDENTITY_OPENID_CONFIG_URL=https://ai.sriphat.com/keycloak/realms/sriphat/.well-known/openid-configuration
|
|
MINIO_IDENTITY_OPENID_CLIENT_ID=minio-client
|
|
MINIO_IDENTITY_OPENID_CLIENT_SECRET=<secret>
|
|
MINIO_IDENTITY_OPENID_CLAIM_NAME=policy
|
|
MINIO_IDENTITY_OPENID_SCOPES=openid,profile,email
|
|
MINIO_IDENTITY_OPENID_REDIRECT_URI=https://ai.sriphat.com/minio-console/oauth_callback
|
|
```
|
|
|
|
**Policy Mapping:** User ใน Keycloak ต้องมี attribute `policy` ที่ map กับ MinIO policy
|
|
|
|
---
|
|
|
|
## Environment Variables
|
|
|
|
```bash
|
|
# Credentials
|
|
MINIO_ROOT_USER=minioadmin
|
|
MINIO_ROOT_PASSWORD=<secret>
|
|
|
|
# URLs
|
|
MINIO_SERVER_URL=https://ai.sriphat.com/minio
|
|
MINIO_BROWSER_REDIRECT_URL=https://ai.sriphat.com/minio-console
|
|
|
|
# Region
|
|
MINIO_REGION=ap-southeast-1
|
|
|
|
# Keycloak SSO
|
|
MINIO_IDENTITY_OPENID_CONFIG_URL=<keycloak-oidc-url>
|
|
MINIO_IDENTITY_OPENID_CLIENT_ID=<client-id>
|
|
MINIO_IDENTITY_OPENID_CLIENT_SECRET=<secret>
|
|
MINIO_IDENTITY_OPENID_CLAIM_NAME=policy
|
|
MINIO_IDENTITY_OPENID_SCOPES=openid,profile,email
|
|
MINIO_IDENTITY_OPENID_REDIRECT_URI=<redirect-uri>
|
|
|
|
TZ=Asia/Bangkok
|
|
```
|
|
|
|
---
|
|
|
|
## Volume Mounts
|
|
|
|
```
|
|
07-minio/
|
|
├── data/ → /data (object storage data)
|
|
└── certs/ → /root/.minio/certs:ro (SSL certificates)
|
|
```
|
|
|
|
---
|
|
|
|
## การใช้งาน MinIO Client (mc)
|
|
|
|
```bash
|
|
# Install
|
|
wget https://dl.min.io/client/mc/release/linux-amd64/mc
|
|
chmod +x mc && sudo mv mc /usr/local/bin/
|
|
|
|
# Config alias
|
|
mc alias set sriphat https://ai.sriphat.com/minio minioadmin <password>
|
|
|
|
# List buckets
|
|
mc ls sriphat
|
|
|
|
# Create bucket
|
|
mc mb sriphat/raw-data
|
|
mc mb sriphat/ml-models
|
|
mc mb sriphat/backups
|
|
mc mb sriphat/reports
|
|
|
|
# Upload
|
|
mc cp data.csv sriphat/raw-data/
|
|
mc cp -r ./models/ sriphat/ml-models/
|
|
|
|
# Set bucket policy
|
|
mc anonymous set none sriphat/raw-data # private
|
|
mc anonymous set download sriphat/public # public read
|
|
```
|
|
|
|
---
|
|
|
|
## Python SDK (boto3)
|
|
|
|
```python
|
|
import boto3
|
|
from botocore.client import Config
|
|
|
|
s3 = boto3.client(
|
|
's3',
|
|
endpoint_url='https://ai.sriphat.com/minio',
|
|
aws_access_key_id='minioadmin',
|
|
aws_secret_access_key='<password>',
|
|
config=Config(signature_version='s3v4'),
|
|
region_name='ap-southeast-1'
|
|
)
|
|
|
|
# Upload file
|
|
s3.upload_file('data.csv', 'raw-data', 'data.csv')
|
|
|
|
# Download file
|
|
s3.download_file('raw-data', 'data.csv', 'local-data.csv')
|
|
|
|
# List objects
|
|
for obj in s3.list_objects_v2(Bucket='raw-data').get('Contents', []):
|
|
print(obj['Key'])
|
|
```
|
|
|
|
---
|
|
|
|
## Recommended Bucket Structure
|
|
|
|
```
|
|
sriphat/
|
|
├── raw-data/ # ข้อมูลดิบจาก Airbyte / HIS
|
|
│ ├── his/
|
|
│ ├── oracle-lab/
|
|
│ └── finance-excel/
|
|
├── processed-data/ # ข้อมูลที่ transform แล้ว
|
|
├── ml-models/ # ML/AI model files
|
|
│ ├── waiting-time/
|
|
│ └── patient-flow/
|
|
├── reports/ # Excel, PDF reports
|
|
├── backups/ # Database backups
|
|
│ └── postgres/
|
|
└── uploads/ # User uploads จาก API Service
|
|
```
|
|
|
|
---
|
|
|
|
## Security
|
|
|
|
```bash
|
|
# สร้าง read-only policy
|
|
cat > readonly-policy.json << 'EOF'
|
|
{
|
|
"Version": "2012-10-17",
|
|
"Statement": [
|
|
{
|
|
"Effect": "Allow",
|
|
"Action": ["s3:GetObject", "s3:ListBucket"],
|
|
"Resource": ["arn:aws:s3:::*"]
|
|
}
|
|
]
|
|
}
|
|
EOF
|
|
|
|
mc admin policy create sriphat readonly-policy readonly-policy.json
|
|
|
|
# Assign policy ให้ user
|
|
mc admin policy attach sriphat readonly-policy --user=analyst-user
|
|
```
|
|
|
|
---
|
|
|
|
## Health Check
|
|
|
|
```bash
|
|
# ตรวจสอบสถานะ
|
|
curl -f http://localhost:9000/minio/health/live
|
|
docker exec minio curl -f http://localhost:9000/minio/health/live
|
|
```
|
|
|
|
---
|
|
|
|
## Backup Strategy
|
|
|
|
```bash
|
|
# Backup data directory
|
|
tar -czf minio-backup-$(date +%Y%m%d).tar.gz 07-minio/data/
|
|
|
|
# Sync to remote
|
|
rsync -avz 07-minio/data/ backup-server:/backups/minio/
|
|
|
|
# Restore
|
|
docker compose down
|
|
tar -xzf minio-backup-20260501.tar.gz
|
|
docker compose up -d
|
|
```
|
|
|
|
---
|
|
|
|
## Keycloak Setup (สำหรับ SSO)
|
|
|
|
ดูรายละเอียดที่ `07-minio/KEYCLOAK_INTEGRATION.md`
|
|
|
|
1. สร้าง Client `minio-client` ใน Keycloak Realm `sriphat`
|
|
2. ตั้งค่า Valid Redirect URIs: `https://ai.sriphat.com/minio-console/oauth_callback`
|
|
3. สร้าง Client Scope `minio-policy`
|
|
4. เพิ่ม User Attribute Mapper `policy`
|
|
5. กำหนด `policy` attribute ให้กับ users ตาม MinIO policies
|
|
|
|
---
|
|
|
|
## Related
|
|
|
|
- [[00-Project-Overview]]
|
|
- [[01-Infrastructure]] (Keycloak SSO)
|
|
- [[07-Security-Strategy]]
|