# Performance Tuning

Adjust throughput, latency, and sampling to fit your workload. For indicative performance numbers, see [Benchmarks](benchmarks.md). For protecting latency under slow sinks, see [Non-blocking Async Logging](../cookbook/non-blocking-async-logging.md).

## Worker count (most impactful)

The `worker_count` setting controls parallel flush processing and has the largest impact on throughput:

| Configuration | Throughput | vs Default |
|---------------|------------|------------|
| 1 worker (default) | ~3,500/sec | baseline |
| 2 workers | ~105,000/sec | **+30x faster** |
| 2 workers + redaction | ~89,000/sec | +26x |

**Recommendation:** Use 2 workers for production workloads. Production-oriented presets (`production`, `fastapi`, `serverless`, `hardened`) default to 2 workers automatically.

```python
from fapilog import LoggerBuilder

# Option 1: Use a production preset (recommended)
logger = LoggerBuilder().with_preset("production").build()

# Option 2: Explicitly set worker count
logger = LoggerBuilder().with_workers(2).build()
```

```bash
# Via environment variable
export FAPILOG_CORE__WORKER_COUNT=2
```

**Why 2 workers is optimal:**
- More than 2 workers shows diminishing returns due to asyncio scheduler overhead
- Queue size barely matters - larger queues actually hurt slightly (memory overhead)
- Workers are the bottleneck with `worker_count=1` (serializes all processing)

**Note on "context switching":** Workers are asyncio tasks running in the dedicated background thread's event loop, not OS threads. There's no OS-level context switching between workers—they run cooperatively. The "overhead" with 3+ workers is the asyncio scheduler managing more tasks within the same thread. See [Execution Modes](execution-modes.md) for details.

**When to use 1 worker:**
- Development/debugging (simpler log ordering)
- Dev preset uses 1 worker by default for this reason

## Queue and batch tuning

```bash
# Throughput-friendly
export FAPILOG_CORE__MAX_QUEUE_SIZE=20000
export FAPILOG_CORE__BATCH_MAX_SIZE=256
export FAPILOG_CORE__BATCH_TIMEOUT_SECONDS=0.25

# Latency-sensitive
export FAPILOG_CORE__MAX_QUEUE_SIZE=5000
export FAPILOG_CORE__BATCH_MAX_SIZE=64
export FAPILOG_CORE__BATCH_TIMEOUT_SECONDS=0.1
export FAPILOG_CORE__DROP_ON_FULL=true
export FAPILOG_CORE__BACKPRESSURE_WAIT_MS=10
```

## Protected levels (priority-aware dropping)

Under queue pressure, fapilog uses priority-aware dropping to protect high-value events. By default, ERROR, CRITICAL, and FATAL events are protected from queue drops.

```python
from fapilog import LoggerBuilder

# Default: ERROR, CRITICAL, FATAL protected
logger = LoggerBuilder().with_preset("production").build()

# Custom: Also protect AUDIT and SECURITY levels
logger = (
    LoggerBuilder()
    .with_protected_levels(["ERROR", "CRITICAL", "FATAL", "AUDIT", "SECURITY"])
    .build()
)

# Disable priority dropping (FIFO behavior)
logger = LoggerBuilder().with_protected_levels([]).build()
```

```bash
# Via environment variable
export FAPILOG_CORE__PROTECTED_LEVELS='["ERROR", "CRITICAL", "FATAL", "AUDIT"]'
```

**How it works:**
- When the queue is full and a protected-level event arrives, an unprotected event is evicted
- Uses O(1) tombstone-based eviction (no queue scanning)
- Maintains temporal ordering within each priority class
- Falls back to normal drop behavior when no eviction candidates exist

**Performance characteristics:**
- Normal enqueue: O(1), no overhead vs standard queue
- Eviction enqueue: O(1), just marks a tombstone
- Dequeue: O(1) amortized, skips tombstones with lazy compaction

**When to customize:**
- Add custom levels (AUDIT, SECURITY) for compliance logging
- Set `[]` to disable priority dropping for strict FIFO behavior
- Leave default for most production workloads

## Adaptive pipeline (automatic tuning)

Instead of manually tuning workers, batch sizes, and queue sizes, the adaptive pipeline monitors queue pressure and adjusts automatically.

```python
from fapilog import LoggerBuilder

# One-line setup with the adaptive preset
logger = LoggerBuilder().with_preset("adaptive").build()

# Or enable adaptive on any preset
logger = (
    LoggerBuilder()
    .with_preset("production")
    .with_adaptive(max_workers=8)
    .build()
)

# Enable batch sizing when using batch-aware sinks (CloudWatch, Loki, PostgreSQL)
logger = (
    LoggerBuilder()
    .with_preset("production")
    .with_adaptive(max_workers=8, batch_sizing=True)
    .add_cloudwatch("/myapp/prod")
    .build()
)
```

```bash
# Via environment variables
export FAPILOG_ADAPTIVE__ENABLED=true
export FAPILOG_ADAPTIVE__MAX_WORKERS=8
export FAPILOG_ADAPTIVE__BATCH_SIZING=true
```

**What it adjusts:**

| Actuator | Normal | Elevated | High | Critical |
|----------|--------|----------|------|----------|
| Workers | Initial (2) | +1 | +2 | Max (8) |
| Batch size (opt-in) | Base (100) | 1.5x | 2x | 4x |
| Queue capacity | Base | 1.5x | 2x | Up to 4x |
| Filter tightening | None | Soft | Medium | Aggressive |

**When to use adaptive vs manual:**
- **Adaptive**: Variable workloads, microservices, deployments where traffic spikes are unpredictable
- **Manual**: Steady-state services where you know exact throughput, latency-critical paths where you want deterministic behavior

See [Adaptive Pipeline](adaptive-pipeline.md) for threshold configuration and detailed tuning.

## Sampling low-severity logs

Use `observability.logging.sampling_rate` to drop a fraction of DEBUG/INFO logs:

```bash
export FAPILOG_OBSERVABILITY__LOGGING__SAMPLING_RATE=0.2  # keep 20% of DEBUG/INFO
```

## Serialization fast-path

Enable `core.serialize_in_flush=true` when sinks support `write_serialized` to reduce per-entry serialization overhead in sinks.

## Metrics

Enable internal metrics to monitor queue depth, drops, flush latency:

```bash
export FAPILOG_CORE__ENABLE_METRICS=true
```