How to Deploy & Use boltdb/bolt

# BoltDB Deploy & Usage Guide

> **Critical Notice**: Bolt is stable but **unmaintained**. The author recommends migrating to the CoreOS fork **[bbolt](https://github.com/coreos/bbolt)** (import path `go.etcd.io/bbolt`) for active projects. This guide covers the original `boltdb/bolt` for legacy systems.

## Prerequisites

- **Go**: Version 1.11 or higher (modules supported). Bolt is pure Go with no CGO dependencies.
- **Operating System**: Linux, macOS, Windows, FreeBSD, OpenBSD, iOS, or Android.
- **Storage**: Sufficient disk space for the database file (Bolt is a single-file database).
- **File System**: Must support file locking (flock). NFS may cause issues with concurrent access.

## Installation

### As a Library Dependency

```bash
# Legacy (unmaintained)
go get github.com/boltdb/bolt

# Recommended active fork (bbolt)
go get go.etcd.io/bbolt

CLI Tool Installation

# Install the bolt command-line utility
go install github.com/boltdb/bolt/cmd/bolt@latest

# Verify installation
bolt version

Configuration

Bolt is configured programmatically via bolt.Options passed to bolt.Open(). There are no configuration files or environment variables consumed by the library itself.

Database Options (bolt.Options)

File Permissions

Bolt databases should use restrictive permissions (owner read/write only):

db, err := bolt.Open("my.db", 0600, &bolt.Options{
    Timeout: 1 * time.Second,
})

Build & Run

Basic Application Structure

package main

import (
    "log"
    "time"
    "github.com/boltdb/bolt"
)

func main() {
    // Open database with timeout to prevent indefinite hangs
    db, err := bolt.Open("my.db", 0600, &bolt.Options{
        Timeout: 1 * time.Second,
    })
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    // Create bucket and write data
    err = db.Update(func(tx *bolt.Tx) error {
        b, err := tx.CreateBucketIfNotExists([]byte("MyBucket"))
        if err != nil {
            return err
        }
        return b.Put([]byte("key"), []byte("value"))
    })
    if err != nil {
        log.Fatal(err)
    }
}

CLI Usage

# Get database info
bolt info my.db

# Get a value
bolt get my.db MyBucket key

# Set a value
bolt set my.db MyBucket key value

# Check database consistency
bolt check my.db

# Backup database (online, read-only transaction)
bolt backup my.db backup.db

# Page-level inspection (advanced debugging)
bolt pages my.db
bolt dump my.db 0  # Dump page 0

Development vs Production Patterns

Development/Bulk Loading (Unsafe):

// Fast import mode - only for initial data loading with crash recovery plan
db, _ := bolt.Open("my.db", 0600, &bolt.Options{NoSync: true})
defer db.Close()
// ... bulk operations ...
// Ensure final sync before closing
db.NoSync = false // Re-enable sync for normal operations

Production:

• Always leave NoSync as false (default)
• Use StrictMode: true only during initial testing, not in production
• Implement proper error handling for ErrTimeout when file is locked

Deployment

Architecture Constraints

Critical: Bolt allows only one read-write process at a time per database file. Multiple processes cannot share the same database file simultaneously. Attempting to open an already open database will cause the second process to hang until the timeout expires.

Deployment Patterns

Single Instance Application (Recommended)

Deploy Bolt with single-instance applications where only one process accesses the database:

FROM golang:1.21-alpine AS builder
WORKDIR /app
COPY . .
RUN go build -o myapp

FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /app/myapp .
VOLUME ["/data"]
ENV DB_PATH=/data/my.db
CMD ["./myapp"]

# docker-compose.yml
version: '3'
services:
  app:
    build: .
    volumes:
      - ./data:/data
    # Ensure only one replica
    deploy:
      replicas: 1

Backup Strategy

Implement online backups using Bolt's WriteTo() method:

func backup(db *bolt.DB, path string) error {
    return db.View(func(tx *bolt.Tx) error {
        return tx.CopyFile(path, 0600)
    })
}

Or use the CLI:

bolt backup production.db "backup-$(date +%Y%m%d).db"

Read-Only Replicas

For read-heavy workloads, copy the database file to read-only instances:

// Read-only instance
db, err := bolt.Open("backup.db", 0600, &bolt.Options{
    ReadOnly: true,
})

Platform-Specific Considerations

• Docker/Kubernetes: Ensure replicas: 1 and use persistent volumes (PV). Bolt is not horizontally scalable.
• Mobile (iOS/Android): Supported. Use appropriate file paths for mobile storage.
• Windows: File locking behaves differently; ensure proper process cleanup on application crashes.
• NFS/SMB: Avoid network filesystems for production; file locking is unreliable.

Troubleshooting

Database Locked / Timeout Errors

Symptom: timeout error or process hangs on bolt.Open().

Cause: Another process holds the file lock.

Solution:

1. Find and terminate the other process accessing the file.
2. Always set Timeout in options to prevent indefinite hangs:

   bolt.Open("my.db", 0600, &bolt.Options{Timeout: 1 * time.Second})
   

Deadlocks

Symptom: Application freezes during transactions.

Cause: Mixing read-write and read-only transactions in the same goroutine, or attempting to start a read-write transaction while holding a read-only transaction.

Solution:

• Never nest db.View() inside db.Update() or vice versa.
• Complete and close transactions before starting new ones of different types.
• Use db.Batch() for high-concurrency write scenarios.

Excessive Database Growth

Symptom: Database file size grows rapidly without corresponding data increase.

Cause: Long-running read transactions prevent page reclamation by the writer.

Solution:

• Keep read transactions short.
• Never hold a transaction open across network calls or user input.
• Use db.Stats() to monitor free pages.

"Key Too Large" or "Value Too Large" Errors

Symptom: ErrKeyTooLarge or ErrValueTooLarge panics.

Limits:

• Max Key Size: 32,768 bytes (32KB)
• Max Value Size: 2,147,483,646 bytes (~2GB)

Solution: Split large values across multiple keys or use external storage with Bolt storing references.

Database Corruption

Symptom: ErrChecksumMismatch or ErrInvalid on open.

Recovery:

1. Check with CLI: bolt check my.db
2. If corrupted, attempt recovery from latest backup.
3. Enable StrictMode during development to catch issues early:

   db.StrictMode = true
   

Memory Issues with Large Databases

Symptom: High memory usage proportional to database size.

Cause: Bolt uses memory-mapped I/O (mmap). The entire database is mapped into virtual memory.

Solution:

• For databases larger than RAM, set WriteFlag to syscall.O_DIRECT when using Tx.WriteTo() to avoid page cache thrashing.
• Use InitialMmapSize to pre-allocate if expecting large growth.

OpenBSD Specifics

On OpenBSD, the NoSync option is ignored and all writes are synchronized regardless, due to lack of unified buffer cache (UBC).

const IgnoreNoSync = runtime.GOOS == "openbsd"