Configuration basics

Everything is one TOML file, plus LUMEN_* environment variable overrides that use __ for nesting, e.g. LUMEN_SERVER__PORT=9090. In the TOML file itself, top-level keys must appear before any [table] header. The exhaustively commented reference is config.example.toml on GitHub.

Section tour

log_format - "pretty" (human-readable, default) or "json" (production).

[server] - the HTTP server: host (bind address; use "0.0.0.0" in a container) and port (must not be 0), body_limit (max request body in bytes), first_token_timeout_ms (how long to wait for the upstream’s first sign of life before failing with LM-3011; for streaming this is time to the first SSE frame, for non-streaming it is the whole upstream call), and sse_heartbeat_ms (idle interval after which a : ping SSE comment keeps proxies from reaping a silent stream).

[auth] - virtual keys, hard budgets, quotas and the usage log. Disabled by default, in which case the gateway is an open proxy with no database at all. See Keys & budgets.

[telemetry] - which x-lumen-metadata keys become Prometheus labels on the token counters. See Usage log.

[resilience] - retries, circuit breaker, timeouts and health checks. Every value is the built-in default and the whole section is optional; see config.example.toml for the full set. Details in Resilience.

[[providers]] / [[providers.models]] - one [[providers]] block per upstream (id, upstream_id, capabilities, modalities, costs, and an optional release_date ISO date that GET /v1/models returns as release_date and as the OpenAI integer created, so clients can sort models by release). See Providers for the full provider matrix and per-provider notes. These are the foundation models: plain upstream bindings. Fallback, split and conditional routing no longer live on them (the per-model fallbacks field was removed).

[[virtual_models]] - public ids that carry routing logic over your foundation models: single, fallback, split and switch strategies, overrides, chat presets, and Jev as a reranker. A virtual model has an id, one capability, a strategy, targets, and optionally fallback_on, preset, a description shown on GET /v1/models (where it appears with "virtual": true) and listed = false to hide it from that list while it stays callable. Foundation and virtual ids share one namespace. See Virtual models. An older config with fallbacks is converted by lumen config migrate.

[image_fetch] - server-side fetching of remote image URLs for multimodal input. See Multimodal input.

API keys

API keys are never written in the config. A provider references the name of the environment variable that holds its key, via api_key_env.

Hot reload

A SIGHUP, a file watch (file mode), an admin config write, or an admin provider-key rotation triggers a reload: the new config is validated, then the provider registry, price table, resilience policy, the runtime-safe [auth] knobs (flush_interval_ms, retention_days), the webhook policy, the image-fetch policy and the tokenizer are atomically swapped.

The boot layer is read once at startup and needs a restart: [server] (bind address, body limit, stream timeouts), log_format, [telemetry] (its label allowlist becomes the Prometheus label set), auth.enabled, auth.db_path, the usage-log channel knobs (usage_channel_capacity, usage_batch_max, usage_flush_ms), [usage_events] (billing usage events for the Meilisearch Lab, ADR 015; also set by LAB_URL and LAB_INSTANCE_ID), and config_source. Details in Deployment.

Where the config lives

By default the whole config is the one TOML file the gateway was started with (config_source = "file"). Setting config_source = "db" moves everything outside the boot layer into the auth database instead, so the gateway can be configured entirely through the admin API with no config file to manage beyond a few boot keys. See Config source modes.

Viewing the live config over the admin API

GET /admin/config returns the current config document, verbatim: in file mode, the config file the gateway booted from; in db mode, the dynamic document stored in the database (an empty string before the first write). It requires the master key AND auth.enabled = true: the whole /admin/* router is only mounted when auth is on, so on a default deployment (where [auth] is disabled) this route does not exist at all and answers 404 rather than 401.

{
  "config": "<raw toml, byte for byte>",
  "hash": "<64 hex chars, BLAKE3>",
  "key_sources": { "openai": "env", "cohere": "stored", "local": "not_required" }
}

config is the document’s exact bytes, never a re-serialisation of the merged in-memory config: Config::load overlays LUMEN_* environment variables on top of it, and showing that merged view would make environment overrides look like document content, and a later write would bake them in permanently. hash is a BLAKE3 content hash of those same bytes, meant to be echoed as If-Match on the PUT /admin/config that applies a new one (ADR 010, ADR 012), so two operators editing at once cannot silently clobber each other.

To read or change one provider or one section without handling the whole document, use the granular endpoints (/admin/config/providers/{name}, /admin/config/{section}) described in Config source modes.

key_sources maps each provider in the dynamic document (the file in file mode, the stored document in db mode) to where its API key resolves from, using the same precedence as startup and hot reload: env (the provider’s api_key_env variable is set in the gateway’s environment, which wins), stored (a key stored via PUT /admin/provider-keys/{name} back-fills an unset variable), missing (no key anywhere and the provider kind needs one), or not_required (a keyless kind such as Ollama, TEI, vLLM, NIM or Bedrock). It reports the source only, never the key. If the document does not parse, key_sources is an empty object; config is still returned verbatim.

Applying a new config over the admin API

PUT /admin/config is the highest-privilege route in the gateway: it can repoint any provider’s base_url (or add a new provider entirely) and thereby redirect customer traffic to a different upstream. The granular provider endpoints can do the same. Master key required, same as every other /admin/* route, and likewise only mounted when auth.enabled = true: with auth off the route is absent and returns 404, not 401.

The If-Match contract:

• Send the submitted document as the raw request body (Content-Type is irrelevant; the body is treated as the document’s new contents).
• Send the hash from a prior GET /admin/config as the If-Match header. A missing If-Match header is rejected with 400 (LM-1001) - the request is malformed like any other missing-required-input case.
• If If-Match does not equal the current document’s hash (someone else applied a change since you last read it, or in file mode someone edited the file directly), the request is rejected with 412 (LM-1004) and nothing is written. GET /admin/config again to see what changed, then re-apply against the fresh hash.

What happens on a successful apply:

1. The candidate is validated exactly like a hot reload would: parsed, merged with LUMEN_* env vars, and used to build a candidate provider registry. A parse failure or a registry-build failure (e.g. a provider missing a required base_url) is rejected with 400 (LM-1001) and nothing is written. The error message names the offending field or setting (e.g. "server.first_token_timeout_ms must not be 0") but never a filesystem path, which is logged server-side instead.
2. A candidate that changes a restart-only (boot-layer) setting is rejected with 400 (LM-1001), naming it: the route only accepts what it can apply immediately. In db mode, a candidate that carries any boot-layer key at all is rejected the same way.
3. The document is persisted atomically. File mode: the current file is copied to a .bak sibling (e.g. lumen.toml.bak) - one generation of history, enough to revert a bad apply by hand - and the new document is renamed into place. DB mode: a new row is added to config_versions (the newest 50 are kept).
4. The hot-reload trigger fires, so the new config takes effect without a restart (see Hot reload). Every setting this route accepts is live once it returns 204.

A rejected apply (400 or 412) is guaranteed to leave the stored document byte-for-byte unchanged and to leave no temporary file behind. Concurrent PUTs are serialised gateway-side, so two operators racing the same pre-apply hash can never both land: exactly one wins, and the other sees a 412 for a hash that moved out from under it, never a silently corrupted mix of the two documents.

Security note: the master key is equivalent to host filesystem access. api_key_env accepts ANY environment variable name, not just ones a provider convention would suggest. Because PUT /admin/config lets a master-key holder add a provider with an attacker-controlled base_url and api_key_env pointed at any variable present in the gateway’s own process environment - LUMEN_MASTER_KEY itself, or any cloud credential the process happens to carry - a single crafted config plus one request to that provider’s model returns the named secret back to the caller as a Bearer token. Before remote config apply existed, reaching this required filesystem write access on the gateway host; with this route, the master key alone is enough. In practice: holding the master key is now equivalent to filesystem write access on the gateway host, plus read access to its entire process environment. Mitigate with a separate master key per gateway, exposing the admin surface on a private network only, and using the read-only config mount opt-out (see Deployment) on any file-mode gateway that does not need remote apply. In db mode the admin API is the only way to change the config, so there is no equivalent opt-out.

Validate before you boot

Run lumen --check-config --config config.toml to validate a config file without starting the server. See Installation.