update configuration docker setup for data platform
This commit is contained in:
jigoong
362
07-minio/KEYCLOAK_INTEGRATION.md
Normal file
362
07-minio/KEYCLOAK_INTEGRATION.md
Normal file
@@ -0,0 +1,362 @@
|
||||
# MinIO Keycloak Integration Guide
|
||||
|
||||
Complete guide for integrating MinIO with Keycloak for SSO authentication.
|
||||
|
||||
## 🎯 Overview
|
||||
|
||||
MinIO supports OpenID Connect (OIDC) authentication, allowing users to log in to MinIO Console using Keycloak credentials. This integration provides:
|
||||
|
||||
- **Single Sign-On (SSO)** - Users authenticate once with Keycloak
|
||||
- **Centralized User Management** - Manage users in Keycloak
|
||||
- **Role-Based Access Control** - Map Keycloak roles to MinIO policies
|
||||
- **Secure Authentication** - OAuth 2.0 / OpenID Connect flow
|
||||
|
||||
## 📋 Prerequisites
|
||||
|
||||
- Keycloak instance running and accessible
|
||||
- MinIO instance running
|
||||
- Admin access to both Keycloak and MinIO
|
||||
|
||||
## 🔧 Setup Steps
|
||||
|
||||
### **Step 1: Create MinIO Client in Keycloak**
|
||||
|
||||
1. **Login to Keycloak Admin Console**
|
||||
```
|
||||
https://ai.sriphat.com/keycloak
|
||||
```
|
||||
|
||||
2. **Select Realm**
|
||||
- Go to your realm (e.g., `sriphat`)
|
||||
|
||||
3. **Create Client**
|
||||
- Navigate to: **Clients** → **Create Client**
|
||||
- **Client ID**: `minio`
|
||||
- **Client Type**: `OpenID Connect`
|
||||
- **Client Protocol**: `openid-connect`
|
||||
- Click **Next**
|
||||
|
||||
4. **Capability Config**
|
||||
- **Client authentication**: `ON`
|
||||
- **Authorization**: `OFF`
|
||||
- **Authentication flow**:
|
||||
- ✅ Standard flow
|
||||
- ✅ Direct access grants
|
||||
- ❌ Implicit flow
|
||||
- ❌ Service accounts roles
|
||||
- Click **Next**
|
||||
|
||||
5. **Login Settings**
|
||||
- **Root URL**: `https://ai.sriphat.com/minio-console`
|
||||
- **Home URL**: `https://ai.sriphat.com/minio-console`
|
||||
- **Valid redirect URIs**:
|
||||
```
|
||||
https://ai.sriphat.com/minio-console/*
|
||||
https://ai.sriphat.com/minio-console/oauth_callback
|
||||
```
|
||||
- **Valid post logout redirect URIs**: `https://ai.sriphat.com/minio-console`
|
||||
- **Web origins**: `https://ai.sriphat.com`
|
||||
- Click **Save**
|
||||
|
||||
6. **Get Client Secret**
|
||||
- Go to **Credentials** tab
|
||||
- Copy the **Client Secret**
|
||||
- Save this for `.env` configuration
|
||||
|
||||
### **Step 2: Create Client Scope for MinIO Policy**
|
||||
|
||||
1. **Create Client Scope**
|
||||
- Navigate to: **Client Scopes** → **Create client scope**
|
||||
- **Name**: `minio-authorization`
|
||||
- **Type**: `Optional`
|
||||
- **Protocol**: `OpenID Connect`
|
||||
- **Display on consent screen**: `OFF`
|
||||
- Click **Save**
|
||||
|
||||
2. **Add Mapper for Policy Claim**
|
||||
- Go to **Mappers** tab
|
||||
- Click **Add mapper** → **By configuration**
|
||||
- Select **User Attribute**
|
||||
- **Name**: `minio-policy`
|
||||
- **User Attribute**: `minio_policy`
|
||||
- **Token Claim Name**: `policy`
|
||||
- **Claim JSON Type**: `String`
|
||||
- **Add to ID token**: `ON`
|
||||
- **Add to access token**: `ON`
|
||||
- **Add to userinfo**: `ON`
|
||||
- Click **Save**
|
||||
|
||||
3. **Assign Scope to MinIO Client**
|
||||
- Go to **Clients** → `minio`
|
||||
- Go to **Client scopes** tab
|
||||
- Click **Add client scope**
|
||||
- Select `minio-authorization`
|
||||
- Choose **Optional**
|
||||
- Click **Add**
|
||||
|
||||
### **Step 3: Create MinIO Policies in Keycloak**
|
||||
|
||||
MinIO uses policies to control access. Common policies:
|
||||
|
||||
- `consoleAdmin` - Full admin access
|
||||
- `readonly` - Read-only access
|
||||
- `readwrite` - Read and write access
|
||||
- `diagnostics` - Diagnostics access
|
||||
|
||||
**Add Policy to Users:**
|
||||
|
||||
1. **Go to Users**
|
||||
- Navigate to: **Users** → Select user
|
||||
|
||||
2. **Add Attribute**
|
||||
- Go to **Attributes** tab
|
||||
- Click **Add attribute**
|
||||
- **Key**: `minio_policy`
|
||||
- **Value**: `consoleAdmin` (or other policy)
|
||||
- Click **Save**
|
||||
|
||||
### **Step 4: Configure MinIO Environment Variables**
|
||||
|
||||
Update `07-minio/.env`:
|
||||
|
||||
```bash
|
||||
# Keycloak Integration
|
||||
MINIO_IDENTITY_OPENID_CONFIG_URL=https://ai.sriphat.com/keycloak/realms/sriphat/.well-known/openid-configuration
|
||||
MINIO_IDENTITY_OPENID_CLIENT_ID=minio
|
||||
MINIO_IDENTITY_OPENID_CLIENT_SECRET=your-client-secret-from-step-1
|
||||
MINIO_IDENTITY_OPENID_CLAIM_NAME=policy
|
||||
MINIO_IDENTITY_OPENID_SCOPES=openid,profile,email,minio-authorization
|
||||
MINIO_IDENTITY_OPENID_REDIRECT_URI=https://ai.sriphat.com/minio-console/oauth_callback
|
||||
```
|
||||
|
||||
### **Step 5: Restart MinIO**
|
||||
|
||||
```bash
|
||||
cd 07-minio
|
||||
docker compose down
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
### **Step 6: Test Authentication**
|
||||
|
||||
1. **Access MinIO Console**
|
||||
```
|
||||
https://ai.sriphat.com/minio-console
|
||||
```
|
||||
|
||||
2. **Click "Login with SSO"**
|
||||
- You'll be redirected to Keycloak
|
||||
- Login with Keycloak credentials
|
||||
- After successful authentication, you'll be redirected back to MinIO Console
|
||||
|
||||
## 🔐 MinIO Policies
|
||||
|
||||
### **Default Policies**
|
||||
|
||||
MinIO comes with built-in policies:
|
||||
|
||||
| Policy | Description |
|
||||
|--------|-------------|
|
||||
| `consoleAdmin` | Full admin access to console and buckets |
|
||||
| `readonly` | Read-only access to buckets |
|
||||
| `readwrite` | Read and write access to buckets |
|
||||
| `diagnostics` | Access to diagnostics and monitoring |
|
||||
| `writeonly` | Write-only access (upload only) |
|
||||
|
||||
### **Custom Policies**
|
||||
|
||||
Create custom policies in MinIO Console or via `mc` CLI:
|
||||
|
||||
```bash
|
||||
# Install mc (MinIO Client)
|
||||
wget https://dl.min.io/client/mc/release/linux-amd64/mc
|
||||
chmod +x mc
|
||||
sudo mv mc /usr/local/bin/
|
||||
|
||||
# Configure mc
|
||||
mc alias set myminio https://ai.sriphat.com/minio minioadmin minioadmin_secure_password_2026
|
||||
|
||||
# Create custom policy
|
||||
cat > custom-policy.json <<EOF
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"s3:GetObject",
|
||||
"s3:PutObject",
|
||||
"s3:DeleteObject"
|
||||
],
|
||||
"Resource": [
|
||||
"arn:aws:s3:::mybucket/*"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
# Add policy to MinIO
|
||||
mc admin policy create myminio custom-policy custom-policy.json
|
||||
```
|
||||
|
||||
## 🔄 Policy Mapping Flow
|
||||
|
||||
```
|
||||
User logs in with Keycloak
|
||||
↓
|
||||
Keycloak returns ID token with 'policy' claim
|
||||
↓
|
||||
MinIO reads 'policy' claim value (e.g., "consoleAdmin")
|
||||
↓
|
||||
MinIO applies corresponding policy to user session
|
||||
↓
|
||||
User has permissions defined by the policy
|
||||
```
|
||||
|
||||
## 🎯 Role-Based Access Example
|
||||
|
||||
### **Scenario: Different User Roles**
|
||||
|
||||
**Admin Users:**
|
||||
```
|
||||
Keycloak User Attribute:
|
||||
minio_policy: consoleAdmin
|
||||
```
|
||||
|
||||
**Data Scientists:**
|
||||
```
|
||||
Keycloak User Attribute:
|
||||
minio_policy: readwrite
|
||||
```
|
||||
|
||||
**Analysts:**
|
||||
```
|
||||
Keycloak User Attribute:
|
||||
minio_policy: readonly
|
||||
```
|
||||
|
||||
## 🐛 Troubleshooting
|
||||
|
||||
### **Issue: "Login with SSO" button not showing**
|
||||
|
||||
**Check:**
|
||||
```bash
|
||||
# Verify environment variables
|
||||
docker exec minio printenv | grep MINIO_IDENTITY_OPENID
|
||||
|
||||
# Check MinIO logs
|
||||
docker logs minio
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
- Ensure all `MINIO_IDENTITY_OPENID_*` variables are set
|
||||
- Restart MinIO container
|
||||
|
||||
### **Issue: Redirect loop after login**
|
||||
|
||||
**Check:**
|
||||
- `MINIO_BROWSER_REDIRECT_URL` matches Keycloak redirect URI
|
||||
- Valid redirect URIs in Keycloak client include `/oauth_callback`
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Update .env
|
||||
MINIO_BROWSER_REDIRECT_URL=https://ai.sriphat.com/minio-console
|
||||
MINIO_IDENTITY_OPENID_REDIRECT_URI=https://ai.sriphat.com/minio-console/oauth_callback
|
||||
```
|
||||
|
||||
### **Issue: User has no permissions after login**
|
||||
|
||||
**Check:**
|
||||
- User has `minio_policy` attribute in Keycloak
|
||||
- Policy name matches MinIO policy exactly (case-sensitive)
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Verify user attribute in Keycloak
|
||||
# Add minio_policy attribute with value: consoleAdmin
|
||||
```
|
||||
|
||||
### **Issue: Cannot access Keycloak config URL**
|
||||
|
||||
**Check:**
|
||||
```bash
|
||||
# Test from MinIO container
|
||||
docker exec minio curl -k https://ai.sriphat.com/keycloak/realms/sriphat/.well-known/openid-configuration
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
- Ensure MinIO container can reach Keycloak
|
||||
- Check network connectivity
|
||||
- Verify Keycloak realm name is correct
|
||||
|
||||
## 📊 Monitoring
|
||||
|
||||
### **Check OpenID Configuration**
|
||||
|
||||
```bash
|
||||
# View current OpenID config
|
||||
docker exec minio mc admin config get myminio identity_openid
|
||||
```
|
||||
|
||||
### **View Active Sessions**
|
||||
|
||||
```bash
|
||||
# List active user sessions
|
||||
docker exec minio mc admin user list myminio
|
||||
```
|
||||
|
||||
### **Audit Logs**
|
||||
|
||||
```bash
|
||||
# Enable audit logging
|
||||
docker exec minio mc admin config set myminio audit_webhook:1 endpoint="http://your-webhook-endpoint"
|
||||
|
||||
# View logs
|
||||
docker logs minio -f
|
||||
```
|
||||
|
||||
## 🔒 Security Best Practices
|
||||
|
||||
1. **Use HTTPS Only**
|
||||
- Always use HTTPS for MinIO and Keycloak
|
||||
- Configure SSL certificates properly
|
||||
|
||||
2. **Rotate Client Secrets**
|
||||
- Periodically rotate Keycloak client secrets
|
||||
- Update MinIO configuration after rotation
|
||||
|
||||
3. **Least Privilege Principle**
|
||||
- Assign minimal required policies to users
|
||||
- Use custom policies for specific use cases
|
||||
|
||||
4. **Monitor Access**
|
||||
- Enable audit logging
|
||||
- Review access logs regularly
|
||||
|
||||
5. **Secure Network**
|
||||
- Use firewall rules to restrict access
|
||||
- Consider VPN for sensitive data
|
||||
|
||||
## 📚 References
|
||||
|
||||
- [MinIO OpenID Connect](https://min.io/docs/minio/linux/operations/external-iam/configure-openid-external-identity-management.html)
|
||||
- [Keycloak OpenID Connect](https://www.keycloak.org/docs/latest/server_admin/#_oidc)
|
||||
- [MinIO IAM Policies](https://min.io/docs/minio/linux/administration/identity-access-management/policy-based-access-control.html)
|
||||
|
||||
## 🎉 Summary
|
||||
|
||||
After completing these steps:
|
||||
- ✅ MinIO integrated with Keycloak SSO
|
||||
- ✅ Users can login with Keycloak credentials
|
||||
- ✅ Role-based access control configured
|
||||
- ✅ Centralized user management in Keycloak
|
||||
- ✅ Secure HTTPS access via Nginx reverse proxy
|
||||
|
||||
**Access MinIO Console:**
|
||||
```
|
||||
https://ai.sriphat.com/minio-console
|
||||
```
|
||||
|
||||
**Login with SSO** → Keycloak authentication → MinIO Console access! 🚀
|
||||
Reference in New Issue
Block a user