sriphat-dataplatform/README-UBUNTU.md

# Sriphat Data Platform - Ubuntu Server Installation Guide

## 📋 System Requirements

- **OS**: Ubuntu Server 20.04 LTS or 22.04 LTS
- **RAM**: Minimum 8GB (16GB recommended)
- **Disk**: 50GB free space
- **CPU**: 4 cores (8 cores recommended)
- **Network**: Static IP recommended

## 🚀 Quick Install (Recommended)

### Option 1: Automated Installation

```bash
# Clone repository
git clone <repository-url> /opt/sriphat-dataplatform
cd /opt/sriphat-dataplatform

# Run install script
bash install.sh
```

Script จะทำให้อัตโนมัติ:
- ติดตั้ง Docker และ Docker Compose
- สร้าง .env.global พร้อม random passwords
- สร้าง backup directory
- รัน services ทั้งหมด

### Option 2: Manual Installation

#### Step 1: Setup Ubuntu Server

```bash
# Update system
sudo apt-get update
sudo apt-get upgrade -y

# Run setup script
sudo bash setup-ubuntu.sh
```

#### Step 2: Logout and Login

```bash
# Logout to apply docker group permissions
exit

# Login again via SSH
ssh user@server
```

#### Step 3: Configure Environment

```bash
cd /opt/sriphat-dataplatform

# Copy and edit .env.global
cp .env.global.example .env.global
nano .env.global

# Update these values:
# - DB_PASSWORD (strong password)
# - KEYCLOAK_ADMIN_PASSWORD
# - SUPERSET_SECRET_KEY
# - ADMIN_SECRET_KEY
# - ADMIN_PASSWORD
```

#### Step 4: Start Services

```bash
# Make scripts executable
chmod +x *.sh
chmod +x 00-network/*.sh

# Start all services
bash start-all.sh
```

## 🔧 Post-Installation

### 1. Check Services Status

```bash
# View running containers
docker ps

# Check logs
docker logs nginx-proxy-manager
docker logs keycloak
docker logs postgres
docker logs apiservice
```

### 2. Access Services

```bash
# Get server IP
hostname -I

# Access points:
# - Nginx Proxy Manager: http://<server-ip>:81
# - Keycloak: http://<server-ip>:8080
```

### 3. Configure Firewall (if needed)

```bash
# Check firewall status
sudo ufw status

# Allow additional ports if needed
sudo ufw allow 8088/tcp  # Superset (if direct access needed)
```

### 4. Setup Domain Names

In Nginx Proxy Manager (port 81):
1. Add Proxy Hosts for each service
2. Configure SSL with Let's Encrypt
3. Point your domain DNS to server IP

## 📦 Directory Structure

```bash
/opt/sriphat-dataplatform/          # Main directory
├── 01-infra/
│   └── data/postgres/              # PostgreSQL data
├── 04-ingestion/
│   └── data/                       # Airbyte data
├── 06-analytics/
│   └── data/                       # Superset data
└── /backups/sriphat-data/          # Backup location
```

## 🔄 Backup Setup

### Automatic Daily Backup

```bash
# Edit crontab
crontab -e

# Add this line (backup at 2 AM daily)
0 2 * * * /opt/sriphat-dataplatform/backup-daily.sh

# Verify cron job
crontab -l
```

### Manual Backup

```bash
# Run backup script
bash backup-daily.sh

# Or backup manually
bash stop-all.sh
sudo tar -czf /backups/sriphat-backup-$(date +%Y%m%d).tar.gz /opt/sriphat-dataplatform
bash start-all.sh
```

## 🛠️ Maintenance Commands

### Start/Stop Services

```bash
# Start all
bash start-all.sh

# Stop all
bash stop-all.sh

# Restart specific service
cd 03-apiservice
docker compose --env-file ../.env.global restart
```

### View Logs

```bash
# All services
docker compose -f 01-infra/docker-compose.yml logs -f

# Specific service
docker logs -f apiservice
docker logs -f postgres
```

### Update Services

```bash
# Stop services
bash stop-all.sh

# Pull latest images
docker compose -f 01-infra/docker-compose.yml pull
docker compose -f 04-ingestion/docker-compose.yml pull
docker compose -f 06-analytics/docker-compose.yml pull

# Rebuild API service
cd 03-apiservice
docker compose --env-file ../.env.global build --no-cache

# Start services
cd ..
bash start-all.sh
```

### Clean Up

```bash
# Remove unused images
docker image prune -a

# Remove unused volumes (careful!)
docker volume prune

# Clean build cache
docker builder prune
```

## 🐛 Troubleshooting

### Docker Permission Denied

```bash
# Add user to docker group
sudo usermod -aG docker $USER

# Logout and login again
exit
```

### Port Already in Use

```bash
# Check what's using the port
sudo netstat -tulpn | grep :80
sudo netstat -tulpn | grep :8080

# Kill process or change port in docker-compose.yml
```

### PostgreSQL Won't Start

```bash
# Check logs
docker logs postgres

# Check permissions
sudo chown -R 999:999 01-infra/data/postgres

# Restart
docker restart postgres
```

### Services Can't Connect to PostgreSQL

```bash
# Check network
docker network inspect shared_data_network

# Verify PostgreSQL is ready
docker exec postgres pg_isready -U postgres

# Restart dependent services
cd 03-apiservice
docker compose --env-file ../.env.global restart
```

### Disk Space Issues

```bash
# Check disk usage
df -h

# Check Docker disk usage
docker system df

# Clean up
docker system prune -a --volumes
```

## 🔒 Security Hardening

### 1. Change Default Passwords

```bash
# Edit .env.global
nano .env.global

# Update all passwords
# Restart services
bash stop-all.sh
bash start-all.sh
```

### 2. Setup SSL

In Nginx Proxy Manager:
1. Add domain
2. Request SSL certificate (Let's Encrypt)
3. Force SSL redirect

### 3. Restrict Firewall

```bash
# Close unnecessary ports after Nginx setup
sudo ufw delete allow 8080/tcp  # Keycloak (access via Nginx only)

# Allow only from specific IPs
sudo ufw allow from 192.168.1.0/24 to any port 81
```

### 4. Enable Fail2ban

```bash
# Install fail2ban
sudo apt-get install fail2ban

# Configure for SSH
sudo systemctl enable fail2ban
sudo systemctl start fail2ban
```

## 📊 Monitoring

### System Resources

```bash
# Real-time monitoring
htop

# Docker stats
docker stats

# Disk usage
df -h
du -sh /opt/sriphat-dataplatform/*
```

### Service Health

```bash
# Check all containers
docker ps -a

# Check specific service health
docker inspect --format='{{.State.Health.Status}}' postgres
```

## 🔄 Migration from Windows

If migrating from Windows development:

```bash
# 1. Backup data on Windows
# (use backup-daily.ps1)

# 2. Copy backup to Ubuntu
scp backup-*.zip user@ubuntu-server:/tmp/

# 3. Extract on Ubuntu
cd /opt/sriphat-dataplatform
unzip /tmp/backup-*.zip

# 4. Fix permissions
sudo chown -R $USER:$USER .
sudo chown -R 999:999 01-infra/data/postgres

# 5. Start services
bash start-all.sh
```

## 📞 Support

For issues:
1. Check logs: `docker logs <container-name>`
2. Verify network: `docker network inspect shared_data_network`
3. Check disk space: `df -h`
4. Review firewall: `sudo ufw status`
5. Consult DEPLOYMENT.md for detailed troubleshooting