update config for limit resouce size
This commit is contained in:
jigoong
400
REMOTE_HOSTS_DOZZLE_SETUP.md
Normal file
400
REMOTE_HOSTS_DOZZLE_SETUP.md
Normal file
@@ -0,0 +1,400 @@
|
||||
# Dozzle Multi-Host Setup Guide
|
||||
|
||||
คู่มือการตั้งค่า Dozzle สำหรับ monitor Docker containers บนหลาย hosts
|
||||
|
||||
## 🏗️ Architecture Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Main Server (Current Host) │
|
||||
│ ├─ Nginx Proxy Manager │
|
||||
│ ├─ Keycloak │
|
||||
│ ├─ PostgreSQL │
|
||||
│ ├─ API Service │
|
||||
│ ├─ Supabase │
|
||||
│ ├─ Superset │
|
||||
│ └─ Dozzle (Main UI) ──────────────┐ │
|
||||
└───────────────────────────────────┼─────────────────────────┘
|
||||
│
|
||||
┌───────────────┴───────────────┐
|
||||
│ │
|
||||
┌───────────▼──────────┐ ┌────────────▼─────────┐
|
||||
│ 192.168.100.9 │ │ 192.168.100.9 │
|
||||
│ Airbyte Host │ │ Airflow Host │
|
||||
│ ├─ Airbyte Services │ │ ├─ Airflow Services │
|
||||
│ └─ Dozzle Agent │ │ └─ Dozzle Agent │
|
||||
│ (Port 7007) │ │ (Port 7008) │
|
||||
└──────────────────────┘ └──────────────────────┘
|
||||
```
|
||||
|
||||
## 📋 Setup Steps
|
||||
|
||||
### **Step 1: ติดตั้ง Dozzle Agent บน Remote Hosts**
|
||||
|
||||
#### **สำหรับ Airbyte Host (192.168.100.9:7007)**
|
||||
|
||||
สร้าง/แก้ไข `docker-compose.yml` ใน Airbyte directory:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
# ... existing Airbyte services ...
|
||||
|
||||
dozzle-agent:
|
||||
image: amir20/dozzle:latest
|
||||
container_name: dozzle-agent-airbyte
|
||||
command: agent
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
ports:
|
||||
- "7007:7007"
|
||||
environment:
|
||||
DOZZLE_LEVEL: info
|
||||
DOZZLE_HOSTNAME: Airbyte Server
|
||||
TZ: Asia/Bangkok
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- airbyte_network # ใช้ network ของ Airbyte
|
||||
```
|
||||
|
||||
**Start agent:**
|
||||
```bash
|
||||
docker compose up -d dozzle-agent
|
||||
```
|
||||
|
||||
#### **สำหรับ Airflow Host (192.168.100.9:7008)**
|
||||
|
||||
สร้าง/แก้ไข `docker-compose.yml` ใน Airflow directory:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
# ... existing Airflow services ...
|
||||
|
||||
dozzle-agent:
|
||||
image: amir20/dozzle:latest
|
||||
container_name: dozzle-agent-airflow
|
||||
command: agent
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
ports:
|
||||
- "7008:7007" # External: 7008, Internal: 7007
|
||||
environment:
|
||||
DOZZLE_LEVEL: info
|
||||
DOZZLE_HOSTNAME: Airflow Server
|
||||
TZ: Asia/Bangkok
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- shared_data_network # ใช้ network ของ Airflow
|
||||
```
|
||||
|
||||
**Start agent:**
|
||||
```bash
|
||||
docker compose up -d dozzle-agent
|
||||
```
|
||||
|
||||
### **Step 2: ตรวจสอบ Agents**
|
||||
|
||||
```bash
|
||||
# ตรวจสอบ Airbyte agent
|
||||
curl http://192.168.100.9:7007/healthcheck
|
||||
|
||||
# ตรวจสอบ Airflow agent
|
||||
curl http://192.168.100.9:7008/healthcheck
|
||||
|
||||
# ดู logs
|
||||
docker logs dozzle-agent-airbyte
|
||||
docker logs dozzle-agent-airflow
|
||||
```
|
||||
|
||||
### **Step 3: Start Dozzle Main UI (Main Server)**
|
||||
|
||||
```bash
|
||||
cd 01-infra
|
||||
docker compose up -d dozzle
|
||||
|
||||
# ตรวจสอบ
|
||||
docker logs dozzle -f
|
||||
```
|
||||
|
||||
### **Step 4: เข้าใช้งาน**
|
||||
|
||||
**Direct access:**
|
||||
```
|
||||
http://localhost:9999/dozzle
|
||||
```
|
||||
|
||||
**ผ่าน Nginx:**
|
||||
```
|
||||
http://ai.sriphat.com/dozzle
|
||||
```
|
||||
|
||||
## 🔧 Configuration Details
|
||||
|
||||
### **Main Server (.env.global)**
|
||||
|
||||
```bash
|
||||
# Dozzle - Docker Log Viewer & Monitoring
|
||||
DOZZLE_PORT=9999
|
||||
DOZZLE_LEVEL=info
|
||||
DOZZLE_BASE=/dozzle
|
||||
DOZZLE_HOSTNAME=Sriphat Main Server
|
||||
DOZZLE_AUTH_PROVIDER=none
|
||||
DOZZLE_RESTART_POLICY=unless-stopped
|
||||
|
||||
# Remote agents: Airbyte and Airflow on 192.168.100.9
|
||||
# Format: host:port,host:port (comma-separated)
|
||||
DOZZLE_REMOTE_AGENT=192.168.100.9:7007,192.168.100.9:7008
|
||||
```
|
||||
|
||||
### **Agent Configuration**
|
||||
|
||||
**Airbyte Agent:**
|
||||
- Port: 7007
|
||||
- Hostname: Airbyte Server
|
||||
- Monitors: Airbyte containers
|
||||
|
||||
**Airflow Agent:**
|
||||
- Port: 7008
|
||||
- Hostname: Airflow Server
|
||||
- Monitors: Airflow containers
|
||||
|
||||
## 🌐 Nginx Configuration
|
||||
|
||||
Dozzle config ถูกเพิ่มใน:
|
||||
- `01-infra/nginx-configs/dozzle.conf`
|
||||
- `01-infra/nginx-configs/complete-example.conf`
|
||||
|
||||
**ตั้งค่าใน Nginx Proxy Manager:**
|
||||
1. ไปที่ Proxy Host → Edit
|
||||
2. Tab "Advanced"
|
||||
3. เพิ่ม Dozzle config จาก `complete-example.conf`
|
||||
|
||||
## 🔍 Features
|
||||
|
||||
### **1. Multi-Host Monitoring**
|
||||
- ✅ ดู logs จาก Main Server
|
||||
- ✅ ดู logs จาก Airbyte Host (192.168.100.9:7007)
|
||||
- ✅ ดู logs จาก Airflow Host (192.168.100.9:7008)
|
||||
- ✅ Switch ระหว่าง hosts ผ่าน dropdown
|
||||
|
||||
### **2. Real-time Log Streaming**
|
||||
- Live log updates
|
||||
- Color-coded logs
|
||||
- JSON formatting
|
||||
- Multi-line grouping
|
||||
|
||||
### **3. Container Management**
|
||||
- View container stats (CPU, Memory, Network)
|
||||
- Start/Stop/Restart containers
|
||||
- Interactive shell access
|
||||
- Container filtering
|
||||
|
||||
### **4. Advanced Features**
|
||||
- Search และ filter logs
|
||||
- Download logs
|
||||
- Multiple container view
|
||||
- SQL-based log querying
|
||||
|
||||
## 🐛 Troubleshooting
|
||||
|
||||
### **Issue: Agent ไม่ปรากฏใน UI**
|
||||
|
||||
**ตรวจสอบ:**
|
||||
```bash
|
||||
# 1. Agent ทำงานหรือไม่
|
||||
docker ps | grep dozzle-agent
|
||||
|
||||
# 2. Port เปิดหรือไม่
|
||||
netstat -tulpn | grep 7007
|
||||
netstat -tulpn | grep 7008
|
||||
|
||||
# 3. Firewall
|
||||
sudo ufw status
|
||||
sudo ufw allow 7007
|
||||
sudo ufw allow 7008
|
||||
|
||||
# 4. Network connectivity
|
||||
ping 192.168.100.9
|
||||
telnet 192.168.100.9 7007
|
||||
telnet 192.168.100.9 7008
|
||||
```
|
||||
|
||||
### **Issue: Connection Refused**
|
||||
|
||||
**สาเหตุ:**
|
||||
- Agent ไม่ทำงาน
|
||||
- Firewall block port
|
||||
- Network ไม่เชื่อมต่อ
|
||||
|
||||
**วิธีแก้:**
|
||||
```bash
|
||||
# Restart agent
|
||||
docker restart dozzle-agent-airbyte
|
||||
docker restart dozzle-agent-airflow
|
||||
|
||||
# ตรวจสอบ logs
|
||||
docker logs dozzle-agent-airbyte
|
||||
docker logs dozzle-agent-airflow
|
||||
|
||||
# ทดสอบ connectivity
|
||||
curl http://192.168.100.9:7007/healthcheck
|
||||
curl http://192.168.100.9:7008/healthcheck
|
||||
```
|
||||
|
||||
### **Issue: Containers ไม่แสดงใน Agent**
|
||||
|
||||
**สาเหตุ:**
|
||||
- Docker socket ไม่ mount
|
||||
- Agent ไม่มี permission
|
||||
|
||||
**วิธีแก้:**
|
||||
```bash
|
||||
# ตรวจสอบ volume mount
|
||||
docker inspect dozzle-agent-airbyte | grep docker.sock
|
||||
|
||||
# ตรวจสอบ permissions
|
||||
ls -la /var/run/docker.sock
|
||||
|
||||
# Restart agent
|
||||
docker restart dozzle-agent-airbyte
|
||||
```
|
||||
|
||||
## 🔐 Security Considerations
|
||||
|
||||
### **1. Network Security**
|
||||
|
||||
**ใช้ Internal Network (แนะนำ):**
|
||||
```yaml
|
||||
# Agent ไม่ expose port ออกภายนอก
|
||||
# ใช้ Docker network แทน
|
||||
dozzle-agent:
|
||||
# ไม่ต้องมี ports section
|
||||
networks:
|
||||
- shared_data_network
|
||||
```
|
||||
|
||||
**Main UI เชื่อมต่อผ่าน network:**
|
||||
```yaml
|
||||
DOZZLE_REMOTE_AGENT=dozzle-agent-airbyte:7007,dozzle-agent-airflow:7007
|
||||
```
|
||||
|
||||
### **2. Firewall Rules**
|
||||
|
||||
```bash
|
||||
# อนุญาตเฉพาะ Main Server
|
||||
sudo ufw allow from <main-server-ip> to any port 7007
|
||||
sudo ufw allow from <main-server-ip> to any port 7008
|
||||
```
|
||||
|
||||
### **3. Authentication**
|
||||
|
||||
**Enable simple auth:**
|
||||
```yaml
|
||||
DOZZLE_AUTH_PROVIDER: simple
|
||||
```
|
||||
|
||||
สร้าง `01-infra/data/dozzle/users.yml`:
|
||||
```yaml
|
||||
users:
|
||||
- name: admin
|
||||
username: admin
|
||||
password: $2a$10$...
|
||||
email: admin@sriphat.com
|
||||
```
|
||||
|
||||
### **4. Read-only Docker Socket**
|
||||
|
||||
```yaml
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
```
|
||||
|
||||
## 📊 Monitoring
|
||||
|
||||
### **Health Checks**
|
||||
|
||||
```bash
|
||||
# Main UI
|
||||
curl http://localhost:9999/dozzle/healthcheck
|
||||
|
||||
# Airbyte Agent
|
||||
curl http://192.168.100.9:7007/healthcheck
|
||||
|
||||
# Airflow Agent
|
||||
curl http://192.168.100.9:7008/healthcheck
|
||||
```
|
||||
|
||||
### **Logs**
|
||||
|
||||
```bash
|
||||
# Main UI
|
||||
docker logs dozzle -f
|
||||
|
||||
# Agents
|
||||
docker logs dozzle-agent-airbyte -f
|
||||
docker logs dozzle-agent-airflow -f
|
||||
```
|
||||
|
||||
## 🎯 Best Practices
|
||||
|
||||
1. **ใช้ Internal Network** - ไม่ expose agent ports ออกภายนอก
|
||||
2. **Enable Authentication** - ใช้ simple auth หรือ forward proxy
|
||||
3. **Monitor Agent Health** - ตั้ง healthcheck และ alerting
|
||||
4. **Backup Configuration** - backup `users.yml` และ `.env` files
|
||||
5. **Update Regularly** - อัพเดท Dozzle image เป็นประจำ
|
||||
6. **Use HTTPS** - ใช้ SSL/TLS สำหรับ production
|
||||
7. **Limit Access** - ใช้ firewall และ access lists
|
||||
|
||||
## 📚 References
|
||||
|
||||
- [Dozzle Documentation](https://dozzle.dev/)
|
||||
- [Agent Mode Guide](https://dozzle.dev/guide/agent)
|
||||
- [Authentication Guide](https://dozzle.dev/guide/authentication)
|
||||
- [Remote Hosts Guide](https://dozzle.dev/guide/remote-hosts)
|
||||
|
||||
## 🔄 Maintenance
|
||||
|
||||
### **Update Dozzle**
|
||||
|
||||
```bash
|
||||
# Main UI
|
||||
cd 01-infra
|
||||
docker compose pull dozzle
|
||||
docker compose up -d dozzle
|
||||
|
||||
# Agents
|
||||
docker pull amir20/dozzle:latest
|
||||
docker restart dozzle-agent-airbyte
|
||||
docker restart dozzle-agent-airflow
|
||||
```
|
||||
|
||||
### **Backup Configuration**
|
||||
|
||||
```bash
|
||||
# Backup .env
|
||||
cp .env.global .env.global.backup
|
||||
|
||||
# Backup users.yml (if using auth)
|
||||
cp 01-infra/data/dozzle/users.yml users.yml.backup
|
||||
```
|
||||
|
||||
## 🎉 Summary
|
||||
|
||||
**ตอนนี้คุณมี:**
|
||||
- ✅ Dozzle Main UI บน Main Server
|
||||
- ✅ Dozzle Agent บน Airbyte Host (192.168.100.9:7007)
|
||||
- ✅ Dozzle Agent บน Airflow Host (192.168.100.9:7008)
|
||||
- ✅ Nginx reverse proxy สำหรับ `/dozzle` subpath
|
||||
- ✅ Multi-host monitoring ผ่าน single UI
|
||||
- ✅ Real-time log streaming จากทุก hosts
|
||||
|
||||
**เข้าใช้งานที่:**
|
||||
```
|
||||
http://ai.sriphat.com/dozzle
|
||||
```
|
||||
|
||||
**Features:**
|
||||
- Monitor logs จาก Main Server, Airbyte, และ Airflow
|
||||
- Real-time streaming
|
||||
- Container stats
|
||||
- Interactive shell
|
||||
- Search และ filter
|
||||
Reference in New Issue
Block a user