# Sriphat Data Platform - Ubuntu Server Installation Guide ## 📋 System Requirements - **OS**: Ubuntu Server 20.04 LTS or 22.04 LTS - **RAM**: Minimum 8GB (16GB recommended) - **Disk**: 50GB free space - **CPU**: 4 cores (8 cores recommended) - **Network**: Static IP recommended ## 🚀 Quick Install (Recommended) ### Option 1: Automated Installation ```bash # Clone repository git clone /opt/sriphat-dataplatform cd /opt/sriphat-dataplatform # Run install script bash install.sh ``` Script จะทำให้อัตโนมัติ: - ติดตั้ง Docker และ Docker Compose - สร้าง .env.global พร้อม random passwords - สร้าง backup directory - รัน services ทั้งหมด ### Option 2: Manual Installation #### Step 1: Setup Ubuntu Server ```bash # Update system sudo apt-get update sudo apt-get upgrade -y # Run setup script sudo bash setup-ubuntu.sh ``` #### Step 2: Logout and Login ```bash # Logout to apply docker group permissions exit # Login again via SSH ssh user@server ``` #### Step 3: Configure Environment ```bash cd /opt/sriphat-dataplatform # Copy and edit .env.global cp .env.global.example .env.global nano .env.global # Update these values: # - DB_PASSWORD (strong password) # - KEYCLOAK_ADMIN_PASSWORD # - SUPERSET_SECRET_KEY # - ADMIN_SECRET_KEY # - ADMIN_PASSWORD ``` #### Step 4: Start Services ```bash # Make scripts executable chmod +x *.sh chmod +x 00-network/*.sh # Start all services bash start-all.sh ``` ## 🔧 Post-Installation ### 1. Check Services Status ```bash # View running containers docker ps # Check logs docker logs nginx-proxy-manager docker logs keycloak docker logs postgres docker logs apiservice ``` ### 2. Access Services ```bash # Get server IP hostname -I # Access points: # - Nginx Proxy Manager: http://:81 # - Keycloak: http://:8080 ``` ### 3. Configure Firewall (if needed) ```bash # Check firewall status sudo ufw status # Allow additional ports if needed sudo ufw allow 8088/tcp # Superset (if direct access needed) ``` ### 4. Setup Domain Names In Nginx Proxy Manager (port 81): 1. Add Proxy Hosts for each service 2. Configure SSL with Let's Encrypt 3. Point your domain DNS to server IP ## 📦 Directory Structure ```bash /opt/sriphat-dataplatform/ # Main directory ├── 01-infra/ │ └── data/postgres/ # PostgreSQL data ├── 04-ingestion/ │ └── data/ # Airbyte data ├── 06-analytics/ │ └── data/ # Superset data └── /backups/sriphat-data/ # Backup location ``` ## 🔄 Backup Setup ### Automatic Daily Backup ```bash # Edit crontab crontab -e # Add this line (backup at 2 AM daily) 0 2 * * * /opt/sriphat-dataplatform/backup-daily.sh # Verify cron job crontab -l ``` ### Manual Backup ```bash # Run backup script bash backup-daily.sh # Or backup manually bash stop-all.sh sudo tar -czf /backups/sriphat-backup-$(date +%Y%m%d).tar.gz /opt/sriphat-dataplatform bash start-all.sh ``` ## 🛠️ Maintenance Commands ### Start/Stop Services ```bash # Start all bash start-all.sh # Stop all bash stop-all.sh # Restart specific service cd 03-apiservice docker compose --env-file ../.env.global restart ``` ### View Logs ```bash # All services docker compose -f 01-infra/docker-compose.yml logs -f # Specific service docker logs -f apiservice docker logs -f postgres ``` ### Update Services ```bash # Stop services bash stop-all.sh # Pull latest images docker compose -f 01-infra/docker-compose.yml pull docker compose -f 04-ingestion/docker-compose.yml pull docker compose -f 06-analytics/docker-compose.yml pull # Rebuild API service cd 03-apiservice docker compose --env-file ../.env.global build --no-cache # Start services cd .. bash start-all.sh ``` ### Clean Up ```bash # Remove unused images docker image prune -a # Remove unused volumes (careful!) docker volume prune # Clean build cache docker builder prune ``` ## 🐛 Troubleshooting ### Docker Permission Denied ```bash # Add user to docker group sudo usermod -aG docker $USER # Logout and login again exit ``` ### Port Already in Use ```bash # Check what's using the port sudo netstat -tulpn | grep :80 sudo netstat -tulpn | grep :8080 # Kill process or change port in docker-compose.yml ``` ### PostgreSQL Won't Start ```bash # Check logs docker logs postgres # Check permissions sudo chown -R 999:999 01-infra/data/postgres # Restart docker restart postgres ``` ### Services Can't Connect to PostgreSQL ```bash # Check network docker network inspect shared_data_network # Verify PostgreSQL is ready docker exec postgres pg_isready -U postgres # Restart dependent services cd 03-apiservice docker compose --env-file ../.env.global restart ``` ### Disk Space Issues ```bash # Check disk usage df -h # Check Docker disk usage docker system df # Clean up docker system prune -a --volumes ``` ## 🔒 Security Hardening ### 1. Change Default Passwords ```bash # Edit .env.global nano .env.global # Update all passwords # Restart services bash stop-all.sh bash start-all.sh ``` ### 2. Setup SSL In Nginx Proxy Manager: 1. Add domain 2. Request SSL certificate (Let's Encrypt) 3. Force SSL redirect ### 3. Restrict Firewall ```bash # Close unnecessary ports after Nginx setup sudo ufw delete allow 8080/tcp # Keycloak (access via Nginx only) # Allow only from specific IPs sudo ufw allow from 192.168.1.0/24 to any port 81 ``` ### 4. Enable Fail2ban ```bash # Install fail2ban sudo apt-get install fail2ban # Configure for SSH sudo systemctl enable fail2ban sudo systemctl start fail2ban ``` ## 📊 Monitoring ### System Resources ```bash # Real-time monitoring htop # Docker stats docker stats # Disk usage df -h du -sh /opt/sriphat-dataplatform/* ``` ### Service Health ```bash # Check all containers docker ps -a # Check specific service health docker inspect --format='{{.State.Health.Status}}' postgres ``` ## 🔄 Migration from Windows If migrating from Windows development: ```bash # 1. Backup data on Windows # (use backup-daily.ps1) # 2. Copy backup to Ubuntu scp backup-*.zip user@ubuntu-server:/tmp/ # 3. Extract on Ubuntu cd /opt/sriphat-dataplatform unzip /tmp/backup-*.zip # 4. Fix permissions sudo chown -R $USER:$USER . sudo chown -R 999:999 01-infra/data/postgres # 5. Start services bash start-all.sh ``` ## 📞 Support For issues: 1. Check logs: `docker logs ` 2. Verify network: `docker network inspect shared_data_network` 3. Check disk space: `df -h` 4. Review firewall: `sudo ufw status` 5. Consult DEPLOYMENT.md for detailed troubleshooting