Codakey
Play
← Back to elastic/elasticsearch

How to Deploy & Use elastic/elasticsearch

# Elasticsearch Deployment and Usage Guide

## 1. Prerequisites

### System Requirements
- **Java**: JDK 21 or JDK 17 (OpenJDK or Oracle JDK). Elasticsearch 8.x requires Java 17 minimum.
- **Memory**: Minimum 4GB RAM (8GB+ recommended for production)
- **Disk**: 20GB free space for build artifacts and data
- **OS**: Linux, macOS, or Windows (Linux recommended for production)

### Required Tools
- **Git**: For cloning the repository
- **Gradle**: The project uses the Gradle Wrapper (`./gradlew`), so local Gradle installation is optional
- **Docker** (optional): For containerized deployment

### Accounts & Licensing
- No account required for basic open-source usage
- Note: This repository contains code under multiple licenses (Elastic License 2.0, GNU Affero GPL v3.0, Server Side Public License v1). Review `LICENSE.txt` before deployment.

## 2. Installation

### Clone the Repository
```bash
git clone https://github.com/elastic/elasticsearch.git
cd elasticsearch
git checkout <version-tag>  # e.g., v8.11.0 (optional, for specific version)

Build from Source

# Build the distribution (creates tar.gz and zip in distribution/archives/)
./gradlew assemble

# Or build specific distribution
./gradlew :distribution:archives:linux-tar:assemble
./gradlew :distribution:archives:darwin-tar:assemble
./gradlew :distribution:archives:windows-zip:assemble

# Skip tests for faster build (tests take 30+ minutes)
./gradlew assemble -x test -x internalClusterTest

Verify Installation

# Check built artifacts
ls distribution/archives/linux-tar/build/distributions/

# Extract and verify
tar -xzf distribution/archives/linux-tar/build/distributions/elasticsearch-*.tar.gz
cd elasticsearch-*/
./bin/elasticsearch --version

3. Configuration

JVM Configuration

Edit config/jvm.options or set environment variables:

# Environment variable approach
export ES_JAVA_OPTS="-Xms4g -Xmx4g -XX:+UseG1GC"

Key JVM settings:

  • -Xms and -Xmx: Set to same value, 50% of available RAM (max 32GB due to compressed OOPs)
  • -XX:+UseG1GC: Recommended garbage collector for Elasticsearch

Node Configuration

Edit config/elasticsearch.yml:

# Cluster settings
cluster.name: my-application
node.name: node-1

# Network settings
network.host: 0.0.0.0  # For development only; use specific IPs in production
http.port: 9200

# Discovery settings (for single-node development)
discovery.type: single-node

# Path settings (optional)
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch

# X-Pack security (enabled by default in 8.x)
xpack.security.enabled: false  # Set to true and configure certs for production

Environment Variables

# Required for production deployment
export ES_PATH_CONF=/path/to/config
export ES_JAVA_OPTS="-Xms4g -Xmx4g"

# Development only
export discovery.type=single-node

4. Build & Run

Development Mode (Single Node)

# Run directly from source (useful for development)
./gradlew run

# Or run the built distribution
./bin/elasticsearch

# Verify running
curl http://localhost:9200

Running Tests

# Run unit tests
./gradlew test

# Run integration tests (requires significant memory)
./gradlew internalClusterTest

# Run specific test class (example from source: AggregationProfilerIT)
./gradlew :server:internalClusterTest --tests "org.elasticsearch.search.profile.aggregation.AggregationProfilerIT"

# Run RRF (Reciprocal Rank Fusion) tests
./gradlew :x-pack:plugin:rank-rrf:internalClusterTest --tests "org.elasticsearch.xpack.rank.rrf.RRFRetrieverBuilderIT"

Production Mode

# Start with configuration
./bin/elasticsearch -Ecluster.name=my-cluster -Enode.name=node-1

# Daemon mode (background)
./bin/elasticsearch -d -p pid

5. Deployment

Docker Deployment (Recommended for Development)

# Build Docker image from source
./gradlew :distribution:docker:buildDockerImage

# Or use official image
docker run -p 9200:9200 -e "discovery.type=single-node" -e "xpack.security.enabled=false" docker.elastic.co/elasticsearch/elasticsearch:8.11.0

Kubernetes Deployment (Production)

Use the official Elasticsearch Operator (ECK):

# elasticsearch.yaml
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: 8.11.0
  nodeSets:
  - name: default
    count: 3
    config:
      node.store.allow_mmap: false

Deploy:

kubectl apply -f https://download.elastic.co/downloads/eck/2.9.0/crds.yaml
kubectl apply -f elasticsearch.yaml

Bare Metal / VM Deployment (Production)

  1. Copy built distribution to servers:
scp distribution/archives/linux-tar/build/distributions/elasticsearch-*.tar.gz user@server:/opt/
  1. Extract and configure:
tar -xzf elasticsearch-*.tar.gz -C /opt/
sudo mv /opt/elasticsearch-* /opt/elasticsearch
sudo useradd elasticsearch
sudo chown -R elasticsearch:elasticsearch /opt/elasticsearch
  1. System configuration (Linux):
# /etc/sysctl.conf
vm.max_map_count=262144
fs.file-max=65536

# Apply
sudo sysctl -p
  1. Create systemd service:
# /etc/systemd/system/elasticsearch.service
[Unit]
Description=Elasticsearch
After=network.target

[Service]
Type=simple
User=elasticsearch
Environment=ES_JAVA_OPTS="-Xms4g -Xmx4g"
Environment=ES_PATH_CONF=/opt/elasticsearch/config
ExecStart=/opt/elasticsearch/bin/elasticsearch
Restart=always

[Install]
WantedBy=multi-user.target

6. Troubleshooting

Bootstrap Checks Failed

Issue: node validation exception on startup Solution:

# Fix virtual memory
sudo sysctl -w vm.max_map_count=262144

# Fix file descriptors
ulimit -n 65536

# Or disable bootstrap checks for development only
./bin/elasticsearch -Enode.attr.testattr=test -Ecluster.routing.allocation.disk.watermark.low=1gb

Out of Memory Errors

Issue: Java heap space or GC overhead limit exceeded Solution:

  • Reduce heap size in config/jvm.options: -Xms2g -Xmx2g
  • Ensure heap is < 50% of physical RAM
  • Check for memory leaks in custom plugins

Port Already in Use

Issue: BindTransportException or Port 9200 already in use Solution:

# Find and kill process
lsof -i :9200
kill -9 <PID>

# Or change ports in elasticsearch.yml
http.port: 9201
transport.port: 9301

Security Certificate Errors (8.x+)

Issue: SSL/TLS handshake failed or security enabled by default Solution:

# Disable security for development (NOT for production)
echo "xpack.security.enabled: false" >> config/elasticsearch.yml

# Or generate certificates for production
./bin/elasticsearch-certutil ca
./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12

Gradle Build Failures

Issue: Could not resolve dependencies or build hangs Solution:

# Clean and rebuild
./gradlew clean
./gradlew assemble --refresh-dependencies

# Skip problematic tests
./gradlew assemble -x test

# Check Java version
java -version  # Should show 17 or 21

Cluster Formation Issues

Issue: Nodes not joining cluster (observed in ShardStateAction logs) Solution:

  • Verify cluster.name matches across nodes
  • Check discovery.seed_hosts includes master-eligible nodes
  • Ensure transport port (9300) is open between nodes
  • Review AbstractScopedSettings errors in logs for invalid settings prefixes

Slow Search/Aggregation Performance

Issue: High latency in aggregations (referencing AggregationProfilerIT patterns) Solution:

  • Enable profiling to diagnose:
GET /_search
{
  "profile": true,
  "aggs": {
    "my_agg": {
      "terms": { "field": "status" }
    }
  }
}
  • Check SubAggCollectionMode settings (BREADTH_FIRST vs DEPTH_FIRST)
  • Increase search.max_buckets if hitting limits (default 65536)
← Project overviewView alternatives to elastic/elasticsearch →