Operating the service

Health endpoints

Each component exposes its own probes. They are not interchangeable: the API being healthy says nothing about whether the sandbox runner can execute code.

The sandbox runner's health check reports whether it can still create and tear down sandbox workspaces: the thing that actually breaks under resource pressure. The Compose file uses GET /api/v2/runtimes as its container healthcheck, which additionally confirms runtimes were discovered.

# Is the whole path working end to end?
curl -s localhost:3112/v1/health
curl -s localhost:2000/api/v2/runtimes | jq 'length'

Metrics

Every component exposes Prometheus metrics at GET /metrics.

On Kubernetes the chart templates a PodMonitor; enable it alongside your Prometheus operator.

Job and execution metrics

Sandbox metrics

File and tool-call metrics

HTTP metrics

codeapi_http_requests_total and codeapi_http_request_duration_seconds, labelled by route and status, on the API and file server; codeapi_sandbox_http_* on the sandbox runner.

What to alert on

Scaling

The tiers scale independently, and they do not scale for the same reasons.

Workers and sandboxes

This is the tier that saturates first. Scale it on queue depth.

# Kubernetes
kubectl scale deployment/codeapi-sandbox-runner --replicas=10

# or via Helm
helm upgrade codeapi ./helm/codeapi \
  --set workerSandbox.replicaCount=10

Per-worker concurrency:

Raising PYTHON_CONCURRENCY beyond 1 is reasonable when jobs are light and memory is plentiful. Watch codeapi_sandbox_execution_duration_seconds for contention as you do.

The API tier

Stateless; scale on CPU and request rate. The chart templates an HPA (api.autoscaling.enabled). Two replicas is a sensible floor for availability.

File server

Scale on file throughput. Usually not the bottleneck: the heavy lifting is in object storage.

Redis and object storage

Shared dependencies. Redis holds the job queue, session pointers, and the egress ledger. Object storage holds files and, when enabled, session snapshots. Neither is scaled by this chart in any serious way: use a managed service in production.

Capacity planning

Roughly:

• Memory per sandbox: LAUNCHER_RAM_MIB (2048 default) per concurrent microVM, plus runner overhead.
• CPU per sandbox: LAUNCHER_VCPUS (2 default) per concurrent microVM.
• Disk for runtimes: a few GB per node for the packages volume, depending on CODEAPI_LANGUAGES.
• Object storage: driven by file retention and, if enabled, persistent session snapshots at up to CODEAPI_SESSION_STATE_MAX_BYTES each.

The Helm defaults request 500m CPU / 1Gi memory and limit at 2000m / 3Gi per worker-sandbox pod.

Rate limits

Applied per principal on the API, backed by Redis so they hold across replicas:

Rejections return 429 with RateLimit-Limit, RateLimit-Remaining, and Retry-After. See configuration.

Tracing

OpenTelemetry tracing is available and off by default:

otel:
  enabled: true
  exporterOtlpEndpoint: "http://collector.observability:4318"

Trace context propagates from the API through the worker into the sandbox execution, so a slow request can be attributed to a specific stage. If you enable it with network policies on, allow the collector: the chart's networkPolicy.otel block does this.

Upgrades

The components are designed to roll independently:

• Capability formats accept both body and header forms during rolling upgrades.
• Manifest verification tolerates the mixed-version window.

Order that minimises surprises: file server and tool call server → egress gateway → sandbox runners → workers → API. Dependencies come up before the things that call them.

Log locations

# Docker Compose
docker compose logs -f api
docker compose logs -f service-worker
docker compose logs -f sandbox-runner

# Kubernetes
kubectl logs -f deployment/codeapi-api
kubectl logs -f deployment/codeapi-service-worker
kubectl logs -f deployment/codeapi-sandbox-runner

Set SANDBOX_LOG_LEVEL=DEBUG for verbose sandbox output when diagnosing execution problems.