/ Engineering · Deep dive
KumoMTA Lua policy: a deep dive into configuration as code.
KumoMTA replaces the static configuration file with a Lua policy script. The shift sounds incremental and is, in practice, a different paradigm. This guide covers the mental model, the event lifecycle, the helper-versus-custom decision, data source integration patterns, the configuration epoch system, performance traps in hot-path code, and the anti-patterns that turn a flexible system into an unstable one.
In short
KumoMTA configuration is a Lua policy script evaluated at startup and re-evaluated when relevant events fire. The script registers handler functions for events like init, get_egress_path_config, get_queue_config, and get_listener_domain, each firing at the appropriate point in the message lifecycle. The bundled policy helpers handle common cases via TOML files so most operators write very little Lua directly. The Lua layer becomes valuable when you need dynamic integration — DKIM keys from HashiCorp Vault, routing decisions from a live database, custom bounce classification. Performance matters because hot-path events fire frequently; cache aggressively and never block.
-
init.lua
The entry point at
/opt/kumomta/etc/policy/init.lua. Registers event handlers; one-time server setup lives in the init handler. -
6 helpers
Bundled policy modules: shaping, queue, sources, dkim_sign, listener_domains, log_hooks. Cover ninety percent of common patterns via TOML files.
-
10 sec poll
Filesystem monitor polls glob-matched config files every ten seconds, hashes their content, increments the configuration epoch when the hash changes.
-
300 sec
Default Lua policy cache lifetime. Refreshes after three hundred seconds or one thousand twenty-four executions, whichever comes first.
/ 01 — Paradigm
Why does KumoMTA use Lua instead of a static configuration file?
Most MTAs you have used express configuration as static text. Postfix reads main.cf and master.cf, PowerMTA reads its directive stanzas, Sendmail has its venerable m4 macros. The values in those files are evaluated once at load time, held in memory for the lifetime of the process, and require either a restart or an explicit reload command when they change. The model is simple, predictable, and old.
KumoMTA breaks that model deliberately. The configuration is a Lua program that registers handler functions for events. When the SMTP listener needs to know whether to accept a relay attempt, KumoMTA fires the get_listener_domain event and runs whatever Lua you registered for it. When a message is about to leave for Gmail, KumoMTA fires get_egress_path_config with the destination context and uses whatever your Lua returns. Configuration becomes a series of function evaluations, not a parsed text blob.
The trade-off is real. Static config is easier to read for a new operator who has never written a line of Lua. Programmable policy is more expressive for the operator who has, and dramatically more so for any deployment that needs live integration with the surrounding stack. KumoMTA bets on the latter audience and mitigates the cost for the former with the policy helpers, which let you operate the system through TOML files without ever touching Lua directly. The deeper payoff shows up in deliverability work specifically: when shaping, routing, and reputation decisions are code rather than static values, they can read live state — a receiver's recent deferral pattern, a tenant's reputation, the time of day — and respond per message, which is exactly the adaptivity that static stanzas cannot express and that high-volume sending increasingly demands.
What the paradigm unlocks
- Late loading: configuration is evaluated when events fire, not at startup. Memory footprint stays lower.
- Live data sources: a handler can read from Vault, Redis, Postgres, or any HTTP endpoint at the moment it needs the value.
- Conditional logic: the script can branch on tenant, campaign, time of day, or any other property the message carries.
- No reload friction: most configuration changes propagate automatically through the epoch system; explicit reloads are rarely needed.
- Composability: Lua's module system lets you organize configuration the way you organize code, with version control and code review applying naturally.
/ 02 — Lifecycle
The events that fire and what each one is for.
Every Lua policy is a collection of handlers registered against named events. KumoMTA fires the appropriate event at each point in the server lifecycle and the message lifecycle. Understanding when each event fires is the difference between policy that performs and policy that drags throughput down.
init Fires once at server startup The one-time setup event. Define spools, start listeners, configure the accounting database, register policy helpers, set the configuration monitor globs. Anything that needs to happen exactly once at startup goes here. Heavy computation in this handler extends startup time linearly, so keep it focused on registration rather than work.
get_listener_domain Fires during each SMTP transaction Called when the SMTP listener accepts a RCPT TO and needs to decide whether to relay the message, treat it as an Out-of-Band bounce, or process it as a Feedback Loop report. The handler receives the destination domain and returns a config object that drives the decision. The listener_domains helper reads this from a TOML file for static cases.
get_queue_config Fires when a message is queued Called for every message as it enters the scheduled queue. Receives the destination domain, the tenant identifier, the campaign identifier, and the routing domain. Returns the queue configuration including which egress pool the message should be routed through. Most multi-tenant routing logic lives here.
get_egress_path_config Fires when connecting to remote Called each time KumoMTA needs to connect to a remote MTA. Receives the destination domain, the chosen egress source, and the site name derived from MX rollup. Returns the egress path config with connection limits, message rates, TLS settings, and timeouts. The shaping helper feeds this from the merged shaping TOML pipeline.
smtp_server_message_received Fires after message body received Called once the full message body has been received by the SMTP listener. The standard place to perform DKIM signing, add headers, classify the message for routing, or reject it based on content rules. For multi-recipient messages, the handler fires once per recipient as part of the DSN processing flow.
delivery_complete Fires after each delivery attempt Called when a delivery attempt completes, success or failure. The handler receives the message context and the result. Typical use cases: emit a webhook to your application, write a custom log line, increment metrics, update an external suppression list. The log_hooks helper provides a structured way to do this without writing the integration from scratch.
/ Interactive — the lifecycle hooks
Where does each decision belong?
\u2014 \u2014 Fires
\u2014
What you control
\u2014
Example use
\u2014
The core lifecycle events KumoMTA exposes, in order. Names and signatures evolve between releases \u2014 confirm against the current KumoMTA documentation for your version.
/ 03 — Helpers
When should you use a policy helper versus writing custom Lua?
KumoMTA ships six policy helpers that cover the common configuration shapes. Each helper reads a TOML or JSON file and registers the appropriate event handlers. The right rule of thumb is to use helpers until you cannot, then drop to custom Lua only where the helpers fall short.
policy-extras.sources Egress sources and pools from sources.toml. Defines sending IPs, EHLO domains, and the pools they group into.
policy-extras.queue Tenant routing, queue retry intervals, tenant header identifiers, mapping from tenant to egress pool. Reads queues.toml.
policy-extras.shaping Traffic shaping rules and TSA automation. Merges defaults, community file, and your overrides into the egress path config.
policy-extras.dkim_sign DKIM signing policy. Reads dkim_data.toml with domain selectors, key file paths, signing policies, and secondary signatures.
policy-extras.listener_domains Which domains the SMTP listener accepts and how it treats them: relay, OOB bounce processing, ARF feedback loop, or reject.
policy-extras.log_hooks Structured integration with webhooks, Kafka, AMQP, or file-based logging. Hooks the delivery_complete event into the destination of your choice.
Stay on the helpers when…
- Your routing decisions are static and fit in a TOML file
- DKIM keys live on disk and rotate on a regular schedule
- Tenant identification is by a single header value
- Log destinations are standard (webhooks, Kafka, file)
- You want config changes to be reviewable by people who do not write Lua
Reach for custom Lua when…
- DKIM keys live in Vault and rotate on demand
- Routing depends on a database lookup or HTTP call
- Bounce classification needs custom logic beyond the IANA defaults
- SMTP authentication validates against a live identity provider
- Per-tenant behavior varies enough that a TOML file would be unwieldy
/ 04 — Integration
Three integration patterns for live data sources in Lua policy.
The patterns below cover the integrations operators reach for most often. Each one solves a real problem the static-config era could not solve cleanly. All three rely on the same principle: read the dynamic data on demand, cache the result, and bump the configuration epoch when the underlying source changes.
Pattern 1 · DKIM keys from HashiCorp Vault
Replace on-disk DKIM private keys with on-demand Vault lookups. The signing key never touches the filesystem, rotation happens transparently when Vault rotates, and compliance posture improves materially.
-- init.lua snippet
local kumo = require "kumo"
-- Cache fetched keys per-domain; bound to avoid leaks
local key_cache = {}
local function get_dkim_key(domain)
if key_cache[domain] then return key_cache[domain] end
local resp = kumo.http.get {
url = "https://vault.internal:8200/v1/secret/data/dkim/" .. domain,
headers = { ["X-Vault-Token"] = os.getenv("VAULT_TOKEN") },
}
local key = kumo.json_parse(resp.body).data.data.private_key
key_cache[domain] = key
return key
end
kumo.on("smtp_server_message_received", function(msg)
local domain = msg:from_header().domain
msg:dkim_sign {
key = get_dkim_key(domain),
selector = "s1",
domain = domain,
}
end)Pattern 2 · Routing decisions from a database
Decide which egress pool a tenant uses based on live database state — current reputation tier, plan level, or any other column. Lookup runs at queue time with TTL-based caching to bound database load.
-- Cached per-tenant lookup with TTL
local tenant_cache = {}
local CACHE_TTL = 60 -- seconds
local function get_tenant_pool(tenant_id)
local entry = tenant_cache[tenant_id]
if entry and (os.time() - entry.ts) < CACHE_TTL then
return entry.pool
end
local row = kumo.pg.query {
connection = "postgres://edp:****@db.internal/routing",
query = "SELECT egress_pool FROM tenants WHERE id = $1",
params = { tenant_id },
}
tenant_cache[tenant_id] = { pool = row[1].egress_pool, ts = os.time() }
return row[1].egress_pool
end
kumo.on("get_queue_config", function(domain, tenant, campaign, routing_domain)
return kumo.make_queue_config { egress_pool = get_tenant_pool(tenant) }
end)Pattern 3 · SMTP auth validated against an identity provider
Validate SMTP AUTH credentials against an external identity provider rather than a local password file. Lets you revoke credentials centrally and apply per-credential rate limits or scope restrictions.
kumo.on("smtp_server_auth_plain", function(authz, authc, password)
local resp = kumo.http.post {
url = "https://idp.internal/api/auth/validate",
body = kumo.json_encode {
username = authc, password = password, service = "smtp",
},
headers = { ["Content-Type"] = "application/json" },
}
if resp.status == 200 then
return true -- accept the auth
end
return false -- reject; KumoMTA will respond with 535
end)/ 05 — Refresh
How does KumoMTA decide when to refresh your configuration?
The configuration epoch is the mechanism KumoMTA uses to detect that something has changed and propagate the change without requiring a full restart. Understanding the epoch system is the difference between configuration updates that take effect predictably and ones that mysteriously do not.
The two refresh strategies
TTL
The object is considered stale after a refresh interval expires, which triggers the corresponding event handler. Simple, predictable, but can lead to periodic busy cycles in systems with many queues because every queue's refresh fires on its own schedule.
Epoch
The object remains valid until the configuration epoch changes. Far fewer speculative refresh calls, but harder to use when pulling from remote data sources because the epoch needs to be bumped explicitly when the remote source changes.
How an epoch bump happens
- Filesystem change: the ten-second poller computes a hash over glob-matched files; a hash change bumps the epoch automatically.
- Lua-triggered: calling
kumo.bump_config_epoch()from your Lua forces an immediate bump. The shaping helper uses this when the TSA daemon pushes updates. - HTTP endpoint: POSTing to
/api/admin/bump-configurationbumps from outside the process. Useful for deployment pipelines.
Why the poll is filesystem-based
KumoMTA intentionally uses simple filesystem polling rather than OS-level file monitoring like inotify. Glob expressions cannot be watched efficiently through inotify, and inotify itself is not reliable inside every container runtime KumoMTA needs to support. The polling cost is negligible — a hash over a dozen TOML files every ten seconds disappears in noise — and the predictability is worth more than the latency saved.
/ 06 — Performance
Performance traps in Lua policy and how to avoid them.
The flexibility of Lua-based configuration comes with the responsibility of writing code that performs at message throughput. Three traps account for most of the performance problems we have debugged in production policy.
Trap 1
Blocking I/O in hot paths
get_egress_path_config and get_queue_config fire constantly. A synchronous database query or HTTP call in those handlers throttles your throughput to the latency of that call. Cache results aggressively, bound the cache, and bump the epoch when the underlying source changes rather than re-fetching each time.
Trap 2
Heavy work in the init event
Init runs once at startup. Anything you do there extends the startup window during which the server cannot accept mail. Reading a few TOML files is fine. Computing a thousand DKIM key fingerprints, populating a large in-memory data structure from a remote API, or running migration logic belongs in deferred initialization or in a separate process entirely.
Trap 3
Unbounded caches that leak
A naive cache that holds every domain ever seen will grow without bound across the lifetime of kumod. After enough running time the process bloats and either gets killed by the OS or starts paging. Bound your caches with a maximum size, evict by LRU or by TTL, and instrument the cache hit ratio so you notice problems before they show up as out-of-memory.
/ 07 — Anti-patterns
Common Lua policy anti-patterns we have seen in production.
Anti-pattern 1 · Writing Lua for everything when helpers exist
Teams comfortable with Lua sometimes skip the policy helpers and write their own implementations of shaping, queue routing, or DKIM signing. The result works but reinvents the wheel and loses the benefit of community-maintained code that updates with KumoMTA releases. Better: use the helpers as the foundation, drop to custom Lua only for behaviors they do not cover.
Anti-pattern 2 · Global mutable state without coordination
Module-level Lua variables that get written from multiple event handlers introduce race conditions and surprise bugs. KumoMTA serializes events on a single Lua VM but does not protect against logical races in your own code. Better: treat the Lua state as immutable per event invocation; persist state to the spool, the accounting DB, or an external store rather than to a module variable.
Anti-pattern 3 · No validation in the deployment pipeline
The validation mode of kumod (kumod --validate) catches malformed TOML, missing files, and shaping warnings before they touch production. Teams that skip it discover problems at deploy time. Better: wire validation into CI so policy changes that cannot validate cannot merge.
Anti-pattern 4 · Hardcoded secrets in init.lua
Database passwords, API tokens, and Vault credentials embedded directly in the policy script end up in your version control history and your container images. Better: read secrets from environment variables that the systemd service injects, or from a secret store accessed via Lua HTTP. Treat init.lua as code that will be reviewed in a pull request by people other than the author.
Anti-pattern 5 · Single monolithic init.lua
A two-thousand-line init.lua that handles every event in one file becomes unreviewable, untestable, and brittle. Lua's require directive works the way it does in any programming language; use it. Better: organize policy into modules by concern — auth.lua, routing.lua, dkim.lua — and require them from init.lua. Each module is small enough to review independently.
/ 08 — FAQ
Questions developers ask while building policy.
Why does KumoMTA use Lua instead of a static configuration file?
Three reasons. Static config files force every decision at load time, with heavy memory consumption and a forced reload for every change. Lua policy is evaluated when the relevant event fires, so configuration loads lazily and can come from anywhere — a TOML file, a database, an HTTP endpoint, HashiCorp Vault. The third reason is composability: imperative scripting expresses conditional logic and iteration that directive-based languages cannot.
When should I use a KumoMTA policy helper versus writing custom Lua?
Use helpers when your configuration fits the pattern they handle. The shaping, queue, sources, dkim_sign, listener_domains, and log_hooks helpers cover ninety percent of deployments and read TOML and JSON files. Write custom Lua when you need a behavior the helpers do not provide — DKIM key resolution from Vault, per-tenant routing that depends on database state, custom bounce classification, anything that needs live integration with your stack.
How often does KumoMTA refresh the Lua policy when I change it?
KumoMTA caches the compiled Lua policy and refreshes it every three hundred seconds or every one thousand twenty-four executions by default. For configuration files referenced by the policy script, KumoMTA monitors them with a file-glob poller that runs every ten seconds and computes a hash. When the hash changes, the configuration epoch increments and the relevant handlers are re-invoked. For external data sources, call kumo.bump_config_epoch from Lua or hit the /api/admin/bump-configuration HTTP endpoint to force an epoch bump.
Can I store DKIM private keys in HashiCorp Vault?
Yes, and this is one of the patterns the KumoMTA team highlights as a reason for the configuration-as-code design. You write a Lua function that fetches the DKIM private key from Vault at the moment the signing event fires, with appropriate caching to avoid hitting Vault on every message. The signing key never touches the filesystem and rotates whenever Vault rotates it.
What performance traps should I avoid in KumoMTA Lua policy?
Three traps. First, blocking I/O in hot-path event handlers like get_egress_path_config or get_queue_config will throttle throughput because those handlers fire frequently. Cache aggressively. Second, complex Lua computation in the init event extends startup linearly; defer anything that does not need to happen exactly once. Third, unbounded data structures in module-level Lua state leak memory across the lifetime of kumod; bound your caches with size limits or TTLs.
How do I validate my Lua policy before deploying it?
KumoMTA ships a validation mode that runs the policy through helper-level integrity checks without starting the SMTP service. The command is kumod followed by the policy path and the validate flag, and it can run concurrently with an active kumod service without conflicting. The validation catches missing files, malformed TOML, shaping rule warnings from the Rust core, and other issues that would otherwise surface at runtime.
/ Next step
Building Lua policy and want a second pair of eyes on the design?
The engineering team at Email Delivery Platform writes KumoMTA Lua policy daily — integrating Vault, Postgres, custom identity providers, and bespoke routing logic. If you are designing a non-trivial policy and want a review before it goes to production, the discovery call ends with a concrete recommendation or with a confirmation that your design is sound.