Windmill Deployment and Usage Guide

1. Prerequisites

Runtime Requirements:

Docker and Docker Compose (for containerized deployment)
Kubernetes cluster (for Helm deployment)
PostgreSQL database (version 12+)
Redis (for caching and job queue)

Development Tools:

Node.js 18+ (for frontend development)
Deno 1.30+ (for CLI and TypeScript runtime)
Python 3.9+ (for Python scripts)
Go 1.19+ (for Go scripts)
Rust (for Rust scripts)
Git (for version control and sync)

Accounts (Optional):

GitHub account (for OAuth/SSO integration)
SMTP provider (for email notifications)
OAuth providers (Google, GitHub, etc.) for authentication

2. Installation

Docker Compose (Recommended for Self-Hosting)

# Clone the repository
git clone https://github.com/windmill-labs/windmill.git
cd windmill

# Start Windmill with Docker Compose
docker compose up -d

Kubernetes (Helm)

# Add the Windmill Helm repository
helm repo add windmill https://windmill-labs.github.io/windmill-helm-charts/
helm repo update

# Install Windmill
helm install windmill windmill/windmill \
  --namespace windmill \
  --create-namespace \
  --values values.yaml

Cloud Providers

Windmill provides deployment guides for:

AWS (ECS, EKS)
Google Cloud (GKE, Cloud Run)
Azure (AKS)
DigitalOcean (Kubernetes)

Check the official documentation for cloud-specific guides.

3. Configuration

Environment Variables

Create a .env file or set these environment variables:

Core Configuration:

# Database
DATABASE_URL=postgresql://user:password@host:5432/windmill
REDIS_URL=redis://redis:6379

# Server
BASE_URL=http://localhost:8000
BASE_INTERNAL_URL=http://localhost:8000
SERVER_SECRET=your-secret-key-here

# External URL (for webhooks and links)
WM_BASE_URL=http://localhost:3000

# Authentication
WM_TOKEN=your-token-here  # For API access
ENABLE_OPENAI=false  # Set to true for AI features

OAuth/SSO Configuration:

# GitHub OAuth
GITHUB_CLIENT_ID=your-client-id
GITHUB_CLIENT_SECRET=your-client-secret

# Google OAuth
GOOGLE_CLIENT_ID=your-client-id
GOOGLE_CLIENT_SECRET=your-client-secret

# SMTP (for email notifications)
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USERNAME=your-email@gmail.com
SMTP_PASSWORD=your-app-password
SMTP_FROM=noreply@yourdomain.com

Advanced Configuration:

# Resource limits
WORKER_GROUP=default
MAX_WORKERS=4
MAX_RAM_PER_WORKER=4096  # 4GB

# S3 for file storage (optional)
S3_BUCKET=your-bucket
S3_REGION=us-east-1
S3_ACCESS_KEY=your-access-key
S3_SECRET_KEY=your-secret-key

Configuration Files

Docker Compose Configuration: The main docker-compose.yml file includes services for:

windmill-api (main API server)
windmill-worker (script execution workers)
postgres (database)
redis (cache and queue)
windmill-lite (optional embedded mode)

Helm Values: Create a values.yaml file for Helm deployments:

postgresql:
  enabled: true
  auth:
    username: windmill
    password: windmill

redis:
  enabled: true

windmill:
  baseUrl: "https://windmill.yourdomain.com"
  databaseUrl: "postgresql://windmill:windmill@postgres:5432/windmill"
  redisUrl: "redis://redis:6379"

4. Build & Run

Local Development Setup

Frontend Only (for UI development):

cd frontend
npm install
npm run dev
# Frontend runs on http://localhost:3000

Backend + Frontend (full development):

# Install dependencies
npm install
pip install -r requirements.txt  # For Python dependencies
cargo build  # For Rust dependencies

# Start all services
docker compose -f docker-compose.dev.yml up

# Or run services individually:
# Start database
docker run -d -p 5432:5432 postgres:15

# Start Redis
docker run -d -p 6379:6379 redis:7

# Start Windmill server
cargo run --bin windmill-server

# Start Windmill worker
cargo run --bin windmill-worker

Using the CLI:

# Install Windmill CLI
deno install -A --global npm:windmill-cli

# Login to your instance
wmill login https://your-windmill-instance.com

# Sync scripts from local directory
wmill sync push ./scripts

# Pull scripts from remote
wmill sync pull ./scripts

Production Build

Docker Images:

# Build all images
docker compose build

# Or build specific services
docker build -t windmill-api -f Dockerfile.api .
docker build -t windmill-worker -f Dockerfile.worker .
docker build -t windmill-frontend -f Dockerfile.frontend .

Standalone Binary (Rust):

# Build from source
cargo build --release --bin windmill-server
cargo build --release --bin windmill-worker

# The binaries will be in target/release/

5. Deployment

Recommended Deployment Platforms

For Small Teams/Single Server:

Docker Compose on a VPS (DigitalOcean, Linode, AWS EC2)
Minimum requirements: 4GB RAM, 2 vCPUs, 50GB storage

For Production/Enterprise:

Kubernetes (self-managed or cloud-managed)
AWS ECS/EKS with RDS for PostgreSQL and ElastiCache for Redis
Google Cloud Run or GKE with Cloud SQL and Memorystore
Azure AKS with Azure Database for PostgreSQL and Azure Cache for Redis

High Availability Setup:

# Example Kubernetes deployment with multiple replicas
apiVersion: apps/v1
kind: Deployment
metadata:
  name: windmill-api
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
  template:
    spec:
      containers:
      - name: windmill-api
        image: ghcr.io/windmill-labs/windmill-api:latest
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: windmill-secrets
              key: database-url
        - name: REDIS_URL
          valueFrom:
            secretKeyRef:
              name: windmill-secrets
              key: redis-url
        resources:
          requests:
            memory: "512Mi"
            cpu: "250m"
          limits:
            memory: "1Gi"
            cpu: "500m"

Scaling Considerations

Vertical Scaling:

Increase MAX_WORKERS based on CPU cores (2 workers per core recommended)
Adjust MAX_RAM_PER_WORKER based on script requirements
Monitor PostgreSQL connection pool (default: 20 connections)

Horizontal Scaling:

Deploy multiple worker instances with different WORKER_GROUP values
Use Redis Cluster for high-availability job queue
Implement database read replicas for heavy read workloads

Storage:

Use S3-compatible storage for large files and artifacts
Configure PostgreSQL with adequate storage and backup strategy
Enable Redis persistence if job queue durability is required

6. Troubleshooting

Common Issues and Solutions

1. Database Connection Issues:

# Check if PostgreSQL is running
docker ps | grep postgres

# Test database connection
psql "postgresql://user:password@host:5432/windmill"

# Common error: "connection refused"
# Solution: Ensure DATABASE_URL is correct and PostgreSQL is accessible

2. Script Execution Failures:

# Check worker logs
docker logs windmill-worker

# Common issues:
# - Missing dependencies: Add requirements.txt for Python or package.json for TypeScript
# - Permission errors: Check resource permissions and variable access
# - Timeout: Increase timeout in script settings or adjust MAX_RAM_PER_WORKER

3. Authentication Problems:

# Verify OAuth configuration
curl -X GET "http://localhost:8000/api/oauth/providers"

# Check environment variables
echo $GITHUB_CLIENT_ID
echo $GITHUB_CLIENT_SECRET

# Test API token
curl -H "Authorization: Bearer $WM_TOKEN" \
  http://localhost:8000/api/whoami

4. Performance Issues:

# Monitor resource usage
docker stats

# Check queue length
redis-cli llen windmill:queue:jobs

# Optimize PostgreSQL
# Add indexes on frequently queried columns
# Increase connection pool size

5. Webhook/Trigger Issues:

# Test webhook endpoint
curl -X POST "http://localhost:8000/api/w/$WORKSPACE_ID/jobs/run_wait_result/p/$PATH" \
  -H "Content-Type: application/json" \
  -d '{"args": {}}'

# Check webhook configuration
# Ensure BASE_URL is correctly set and accessible from external sources

6. File Storage Problems:

# Test S3 connectivity (if using S3)
aws s3 ls s3://$S3_BUCKET --endpoint-url=$S3_ENDPOINT

# Check file permissions
# Ensure worker has write access to storage directory

7. Getting Help:

Check the official documentation
Join the Discord community
Search existing GitHub issues
Enable debug logging: LOG_LEVEL=debug in environment variables

Logs and Monitoring

Access Logs:

# Docker Compose
docker compose logs -f windmill-api
docker compose logs -f windmill-worker

# Kubernetes
kubectl logs deployment/windmill-api -f
kubectl logs deployment/windmill-worker -f

Health Checks:

# API health endpoint
curl http://localhost:8000/health

# Worker health
curl http://localhost:8000/api/workers/health

Metrics: Windmill exposes Prometheus metrics at /metrics endpoint. Set up monitoring with:

Prometheus for metrics collection
Grafana for visualization
AlertManager for notifications

How to Deploy & Use windmill