sriphat-dataplatform/README-UBUNTU.md

Sriphat Data Platform - Ubuntu Server Installation Guide

📋 System Requirements

• OS: Ubuntu Server 20.04 LTS or 22.04 LTS
• RAM: Minimum 8GB (16GB recommended)
• Disk: 50GB free space
• CPU: 4 cores (8 cores recommended)
• Network: Static IP recommended

🚀 Quick Install (Recommended)

Option 1: Automated Installation

# Clone repository
git clone <repository-url> /opt/sriphat-dataplatform
cd /opt/sriphat-dataplatform

# Run install script
bash install.sh

Script จะทำให้อัตโนมัติ:

• ติดตั้ง Docker และ Docker Compose
• สร้าง .env.global พร้อม random passwords
• สร้าง backup directory
• รัน services ทั้งหมด

Option 2: Manual Installation

Step 1: Setup Ubuntu Server

# Update system
sudo apt-get update
sudo apt-get upgrade -y

# Run setup script
sudo bash setup-ubuntu.sh

Step 2: Logout and Login

# Logout to apply docker group permissions
exit

# Login again via SSH
ssh user@server

Step 3: Configure Environment

cd /opt/sriphat-dataplatform

# Copy and edit .env.global
cp .env.global.example .env.global
nano .env.global

# Update these values:
# - DB_PASSWORD (strong password)
# - KEYCLOAK_ADMIN_PASSWORD
# - SUPERSET_SECRET_KEY
# - ADMIN_SECRET_KEY
# - ADMIN_PASSWORD

Step 4: Start Services

# Make scripts executable
chmod +x *.sh
chmod +x 00-network/*.sh

# Start all services
bash start-all.sh

🔧 Post-Installation

1. Check Services Status

# View running containers
docker ps

# Check logs
docker logs nginx-proxy-manager
docker logs keycloak
docker logs postgres
docker logs apiservice

2. Access Services

# Get server IP
hostname -I

# Access points:
# - Nginx Proxy Manager: http://<server-ip>:8021
# - Keycloak: http://<server-ip>:8080
# - Airbyte: http://<server-ip>:8000

3. Configure Firewall (if needed)

# Check firewall status
sudo ufw status

# Allow additional ports if needed
sudo ufw allow 8088/tcp  # Superset (if direct access needed)

4. Setup Domain Names

In Nginx Proxy Manager (port 81):

1. Add Proxy Hosts for each service
2. Configure SSL with Let's Encrypt
3. Point your domain DNS to server IP

📦 Directory Structure

/opt/sriphat-dataplatform/          # Main directory
├── 01-infra/
│   └── data/postgres/              # PostgreSQL data
├── 04-ingestion/
│   └── data/                       # Airbyte data
├── 06-analytics/
│   └── data/                       # Superset data
└── /backups/sriphat-data/          # Backup location

🔄 Backup Setup

Automatic Daily Backup

# Edit crontab
crontab -e

# Add this line (backup at 2 AM daily)
0 2 * * * /opt/sriphat-dataplatform/backup-daily.sh

# Verify cron job
crontab -l

Manual Backup

# Run backup script
bash backup-daily.sh

# Or backup manually
bash stop-all.sh
sudo tar -czf /backups/sriphat-backup-$(date +%Y%m%d).tar.gz /opt/sriphat-dataplatform
bash start-all.sh

🛠️ Maintenance Commands

Start/Stop Services

# Start all
bash start-all.sh

# Stop all
bash stop-all.sh

# Restart specific service
cd 03-apiservice
docker compose --env-file ../.env.global restart

View Logs

# All services
docker compose -f 01-infra/docker-compose.yml logs -f

# Specific service
docker logs -f apiservice
docker logs -f postgres

Update Services

# Stop services
bash stop-all.sh

# Pull latest images
docker compose -f 01-infra/docker-compose.yml pull
docker compose -f 04-ingestion/docker-compose.yml pull
docker compose -f 06-analytics/docker-compose.yml pull

# Rebuild API service
cd 03-apiservice
docker compose --env-file ../.env.global build --no-cache

# Start services
cd ..
bash start-all.sh

Clean Up

# Remove unused images
docker image prune -a

# Remove unused volumes (careful!)
docker volume prune

# Clean build cache
docker builder prune

🐛 Troubleshooting

Docker Permission Denied

# Add user to docker group
sudo usermod -aG docker $USER

# Logout and login again
exit

Port Already in Use

# Check what's using the port
sudo netstat -tulpn | grep :80
sudo netstat -tulpn | grep :8080

# Kill process or change port in docker-compose.yml

PostgreSQL Won't Start

# Check logs
docker logs postgres

# Check permissions
sudo chown -R 999:999 01-infra/data/postgres

# Restart
docker restart postgres

Services Can't Connect to PostgreSQL

# Check network
docker network inspect shared_data_network

# Verify PostgreSQL is ready
docker exec postgres pg_isready -U postgres

# Restart dependent services
cd 03-apiservice
docker compose --env-file ../.env.global restart

Disk Space Issues

# Check disk usage
df -h

# Check Docker disk usage
docker system df

# Clean up
docker system prune -a --volumes

🔒 Security Hardening

1. Change Default Passwords

# Edit .env.global
nano .env.global

# Update all passwords
# Restart services
bash stop-all.sh
bash start-all.sh

2. Setup SSL

In Nginx Proxy Manager:

1. Add domain
2. Request SSL certificate (Let's Encrypt)
3. Force SSL redirect

3. Restrict Firewall

# Close unnecessary ports after Nginx setup
sudo ufw delete allow 8080/tcp  # Keycloak (access via Nginx only)

# Allow only from specific IPs
sudo ufw allow from 192.168.1.0/24 to any port 81

4. Enable Fail2ban

# Install fail2ban
sudo apt-get install fail2ban

# Configure for SSH
sudo systemctl enable fail2ban
sudo systemctl start fail2ban

📊 Monitoring

System Resources

# Real-time monitoring
htop

# Docker stats
docker stats

# Disk usage
df -h
du -sh /opt/sriphat-dataplatform/*

Service Health

# Check all containers
docker ps -a

# Check specific service health
docker inspect --format='{{.State.Health.Status}}' postgres

🔄 Migration from Windows

If migrating from Windows development:

# 1. Backup data on Windows
# (use backup-daily.ps1)

# 2. Copy backup to Ubuntu
scp backup-*.zip user@ubuntu-server:/tmp/

# 3. Extract on Ubuntu
cd /opt/sriphat-dataplatform
unzip /tmp/backup-*.zip

# 4. Fix permissions
sudo chown -R $USER:$USER .
sudo chown -R 999:999 01-infra/data/postgres

# 5. Start services
bash start-all.sh

📞 Support

For issues:

1. Check logs: docker logs <container-name>
2. Verify network: docker network inspect shared_data_network
3. Check disk space: df -h
4. Review firewall: sudo ufw status
5. Consult DEPLOYMENT.md for detailed troubleshooting