The 0ms-zombie: a TOCTOU class of bug in AI-agent turn persistence
Two-phase write + process death = a row that lies about its own state. We call it the 0ms-zombie. Defines the bug class formally and ranks four mitigations by structural depth, from frontend clamp to DB CHECK constraint.
AppX team ·
An AI agent loop produces a turn. The turn has a beginning ("user sent a message"), some middle ("the model is thinking, calling tools, streaming partials"), and an end ("here is the final outcome — success, failure, or cancellation"). Most agent persistence designs split this into two writes: insert a placeholder row when the turn starts, merge the terminal state when the turn ends.
That two-phase write has a structural failure window. We call the surviving artifact the 0ms-zombie.
The bug shape
sequenceDiagram
participant W as Worker
participant DB as Database
W->>DB: phase 1 — INSERT turn (status: running, 0ms)
Note over W: process dies (SIGKILL / deploy / OOM)
W--xDB: phase 2 — UPDATE to final state never runs
Note over DB: row says "running" forever —<br/>a turn that lies about its own state
The user sees "Done · 0ms" next to an assistant message that says nothing. Click into it: empty body, no tool calls, no error, no reasoning. The UI is convinced the turn completed successfully and took zero milliseconds.
It didn't complete. The process died.
Here is the lifecycle the writer code assumes:
INSERT INTO agent_turns (id, user_id, started_at) VALUES (...)— placeholder row,outcome IS NULL,message = ''.- Agent loop runs. Tools fire. Tokens stream. Eventually a terminal event arrives.
UPDATE agent_turns SET outcome = 'success', message = ?, completed_at = NOW(), duration_ms = ? WHERE id = ?— terminal merge.
Between step 1 and step 3, the world can change. A deploy sends SIGKILL. The OOM killer reaps the worker. A node loses its lease. The TCP connection mid-UPDATE resets. Any of these and the row stays in placeholder shape forever: outcome IS NULL, message = '', completed_at IS NULL.
The frontend hydration path, written months later by someone reasonable, defaults unknown-outcome rows to a renderable shape:
const completedAt = row.completedAt ?? row.startedAt;
const durationMs = completedAt.getTime() - row.startedAt.getTime();
const outcome = row.outcome ?? 'complete';
Now the row says "Done · 0ms". The lie compounds: the data shape said "I don't know what happened," the hydrator said "default to happy path," the UI rendered it confidently. No single line is wrong. The bug is in the join.
Why this is a class
This is a TOCTOU bug in turn persistence: between time-of-insert and time-of-terminal-merge, the world can change, and the code assumes it won't.
It is not a bug in any single call site. It is a property of the data shape. Anywhere you see this pattern, you have the same bug:
- A background job framework that does
INSERT job(status='pending')thenUPDATE job SET status='done'after work runs. - A chat UI that inserts an empty assistant message and patches its body as tokens stream.
- An LLM tool-call loop that surfaces "thinking..." then "result" as separate persistence steps.
- A workflow engine recording state transitions where the "in-progress" row is supposed to be replaced by the "complete" row.
All of these have a window where the "in-progress" record survives a process death and outlives the work it was supposed to describe. The window is small. It does not stay small at scale: deploys, OOMs, and crashes are Poisson processes, not edge cases.
The hardest part of seeing this clearly is that no individual layer is doing anything wrong. The insert is correct. The update is correct. The hydrator is defensive in the way you teach junior engineers to be defensive. The bug only exists in the gap between them, which is exactly where no single code review will catch it.
The 4-layer mitigation stack
Rank-ordered cheapest to deepest. Ship layer 1 today, layer 4 over a sprint. Each layer makes the next one cheaper.
1. Frontend clamp — cosmetic, instant
const durationMs = completedAt && completedAt > startedAt
? completedAt.getTime() - startedAt.getTime()
: null;
Cost: one ternary. Time-to-ship: minutes. Coverage: hides the "0ms" lie from users while you fix the real thing.
This layer does nothing about the underlying corruption. It clamps the visible symptom. Ship it because the user-facing artifact is embarrassing, but do not stop here. Cosmetic fixes accumulate into a UI that lies about a database that lies about itself.
2. Backend sweeper — bounded safety belt
A periodic query that finds zombies and reaps them.
// every 30s
const stale = await db.query(`
SELECT id FROM agent_turns
WHERE outcome IS NULL
AND message = ''
AND created_at < NOW() - INTERVAL 60 SECOND
`);
for (const row of stale) {
await db.execute(`
UPDATE agent_turns
SET outcome = 'failed',
message = 'Turn interrupted before completion',
completed_at = NOW(),
duration_ms = TIMESTAMPDIFF(MICROSECOND, started_at, NOW()) / 1000
WHERE id = ? AND outcome IS NULL
`, [row.id]);
}
Cost: one indexed query per sweep interval. Time-to-ship: a day. Coverage: every zombie eventually gets a real outcome, bounded by the sweep interval.
The key property: the sweeper is your SIGKILL handler. SIGTERM is polite; you can register cleanup. SIGKILL skips your handlers entirely, and so do OOM kills, kernel panics, node failures, and docker kill -9. Any cleanup that has to run during process death is unreliable by construction. The sweeper runs in a fresh process and treats orphans as a class.
Cost math: a 5-minute sweep means a user can see a zombie for up to 5 minutes. A 30-second sweep is cheap on a properly-indexed turns table. Match the interval to your error budget.
3. Single-row-per-turn invariant — structural
Change the schema so the placeholder write is the terminal write. There is no second row to lose.
-- BEFORE: two writes, terminal state implicit
CREATE TABLE agent_turns (
id BINARY(16) PRIMARY KEY,
user_id BINARY(16) NOT NULL,
message TEXT NOT NULL DEFAULT '',
outcome VARCHAR(16) NULL, -- NULL means "in progress"
started_at TIMESTAMP(3) NOT NULL,
completed_at TIMESTAMP(3) NULL,
duration_ms INT NULL
);
-- AFTER: outcome is always set; 'pending' is a real state
CREATE TABLE agent_turns (
id BINARY(16) PRIMARY KEY,
user_id BINARY(16) NOT NULL,
message TEXT NOT NULL DEFAULT '',
outcome ENUM('pending','success','failed','cancelled')
NOT NULL DEFAULT 'pending',
started_at TIMESTAMP(3) NOT NULL,
completed_at TIMESTAMP(3) NULL,
duration_ms INT NULL,
CONSTRAINT terminal_coherence CHECK (
outcome = 'pending'
OR (completed_at IS NOT NULL AND duration_ms IS NOT NULL)
)
);
Cost: a schema migration, a writer-code rewrite, and a backfill (UPDATE agent_turns SET outcome = 'pending' WHERE outcome IS NULL). Time-to-ship: a sprint, because the writer code lives in five places and the test suite has assumptions baked into it.
This is the hardest layer to retrofit because it changes the contract that every writer relies on. We did it in waves; the load-bearing wave was adding the pending enum value with a default. Once that exists, every existing writer becomes correct automatically — their inserts now produce outcome='pending' rows instead of outcome=NULL rows — and the hydrator can distinguish "we know this is in progress" from "we don't know what this is."
NULL-as-state was the original sin. NULL means unknown. A row in progress is not unknown; it has a known state called "running." Model it.
4. DB CHECK constraint — the wall
The constraint in the schema above (terminal_coherence) is the deepest layer. It says: the database will refuse to accept any row that claims to be terminal but has no timing. No application code, no future engineer, no parallel writer can produce a 0ms-zombie, because the row literally cannot be written.
Cost: zero runtime. One schema change. The constraint is enforced on every INSERT and UPDATE; the cost is amortized into normal write path and is too small to measure.
This is the layer that proves the bug class is closed. The other three layers are defenses. This one is a mathematical guarantee.
How to find this in your own system
If you have a turns table, a jobs table, a chat-messages table, or any table that records "this operation started" and "this operation ended" in separate writes, run this query:
SELECT id, created_at,
NOW() - created_at AS age
FROM agent_turns
WHERE outcome IS NULL
AND message = ''
AND created_at < NOW() - INTERVAL '5 MINUTES';
Adjust the predicates to match your schema's "in-progress" shape — the universal pattern is no terminal field set AND age > plausible-completion-time. Anything this query returns is a zombie. If it returns zero, your insert-then-merge writer has been lucky so far. Run it again after your next deploy.
A second tell, in the frontend code: search for the string ?? 'complete', ?? 'success', ?? row.startedAt, or any defaulting-of-unknown-to-happy-path on hydration. If the data layer can produce NULLs and the view layer defaults them to terminal, you have the same bug shape even if you have not seen the symptom yet.
Takeaways
- Two-phase persistence (insert placeholder → merge terminal) is a TOCTOU pattern. Treat the gap as adversarial — deploys and OOMs will land in it.
NULLis not a state. A turn that is in progress has a state called "in progress." Name itpendingand makeoutcomeNOT NULL.- A frontend clamp hides the symptom. A backend sweeper bounds it. A schema invariant eliminates it. A CHECK constraint proves it is eliminated. Ship them in that order; do not stop at the clamp.
- The sweeper is your real SIGKILL handler. Anything that has to run during process death is theater.
- If you cannot draw a state diagram where every node has an explicit name, your database is going to invent one for you. It will be called
NULL, and it will be wrong.