Forge deep-dive: the warm pool that gets a user from chat to live phone preview in under 2 seconds
Sequel to why-we-built-forge. Sub-2-second cold-to-QR is impossible if every request spawns a fresh container. The warm pool primitive: pre-provisioned idle sandboxes, atomic claim, load-aware sizing, ~80 containers per 24GB VPS.
AppX team ·
A user types "build me a meditation timer" into AppX. Roughly two seconds later, their phone is holding a QR code, Expo Go has scanned it, and the generated app is rendering. Two seconds is not where you land by default — it is where you land after you decide a number, refuse to ship anything slower, and rebuild the stack underneath the chat box until it cooperates.
This post is about the single primitive carrying most of that weight: the warm pool.
If you haven't read Why we built Forge, start there. That sibling post explains what Forge is — AppX's per-user app sandbox orchestrator — and why we built it instead of buying E2B, Modal, Daytona, or Replit.
The cold-start tax
| Cold start (no pool) | Warm pool | |
|---|---|---|
| What the user waits for | container create + deps + bundler boot | a ready container, already warm |
| Felt latency | tens of seconds | seconds |
| When the cost is paid | on the user's request | ahead of time, in the background |
If every "generate" click spawned a fresh sandbox, the user would wait through image pull, container start, Node boot, Expo CLI init, Metro warmup, and the dependency graph hydrating in memory. Six to eight seconds on a good day. Most of it is identical for every project AppX has ever served — the only thing that varies is the handful of generated source files at the end.
Six seconds is not "slow." Six seconds is a different product. The chat-to-phone loop stops feeling like magic and starts feeling like a build pipeline. Users tab away. The thing that makes AppX feel like a toy in the good sense is that the phone lights up before the user has put it down. You cannot get there if you pay the warmup tax on every request.
The pool primitive
The trick is older than container orchestration: pre-warm. Forge maintains a fleet of N idle sandboxes — fully booted, Metro running, dependency graph in memory, waiting for source files. When the user clicks generate, the backend claims one, writes the AI-generated files in, and hands back its preview URL. Metro's file watcher catches the change, runs an incremental rebuild, and pushes a fresh bundle to Expo Go.
The user pays roughly one hundred milliseconds for the file write plus Metro's incremental cycle, instead of seven seconds for the entire cold start. The pool is the only reason sub-two-second works. Everything below is the engineering required to keep it honest.
Load-aware pool sizing
Static pool sizes are a trap. Pick five and you run dry the moment a Hacker News spike arrives. Pick fifty and you pay for fifty containers of RAM on a sleepy Wednesday afternoon.
So Forge sizes from recent demand. A controller loop runs every few seconds, looks at the claim rate over a short window and the number of warm slots currently idle, and computes a target.
def compute_pool_target(claim_rate_per_min, idle_warm, in_flight):
demand = claim_rate_per_min * 1.0
buffer = max(MIN_BUFFER, claim_rate_per_min * 0.25)
target = math.ceil(demand + buffer)
target = max(MIN_POOL, min(MAX_POOL, target))
deficit = target - (idle_warm + in_flight)
return max(0, deficit)
When the deficit is positive, the controller starts more sandboxes. When idle slots outnumber the target, the oldest age out. Deliberately boring — a smoothed estimator plus floor and ceiling. Fancy autoscalers are easy to write and hard to debug when they misbehave at 3am.
The state machine
stateDiagram-v2
[*] --> warming
warming --> warm: bundler ready
warm --> claimed: user arrives (atomic UPDATE)
claimed --> running: code pushed
running --> sleeping: idle timeout
sleeping --> running: wake
warm --> [*]: swept if orphaned
Every slot moves through a fixed lifecycle:
+---> error -----+
| |
provisioning | v
| | destroyed
v | ^
warm ----+ |
| |
v |
claimed -> running -> sleeping
warm means Metro is up and the sandbox is willing to accept files. claimed is the brief window between "backend chose this slot" and "backend has written files in." running means a project is live on it. sleeping is the polite shutdown when the user disconnects but we expect them back. error and destroyed are terminal.
Transitions are driven by webhooks from the node agent — every state change reported the instant it happens. A reconciler enforces invariants: a slot marked warm that fails a health probe is demoted to error and a replacement is provisioned. The reconciler exists because the world lies — containers OOM, hosts reboot, network blips eat webhooks. Without it, the pool drifts from reality. With it, the pool heals on its own within a minute.
The claim race (atomic UPDATE)
Two simultaneous generation requests arrive. Both look at the pool. Both pick the same warm slot. Catastrophe — two users staring at one another's app.
The fix is the oldest trick in the relational handbook: make the claim atomic in the database.
UPDATE sandboxes
SET state = 'claimed',
project_id = $1,
claimed_at = NOW()
WHERE id = $2
AND state = 'warm';
Row-level lock. The state = 'warm' predicate is the entire correctness argument. If two requests race, one updates a row and gets affected_rows = 1. The other matches zero rows — the slot was already taken in the microsecond between SELECT and UPDATE. The loser retries against a different warm slot, or surfaces a "warming up" state if the pool is empty.
One query. It replaces an entire category of locking machinery an earlier version of this code tried to introduce. The orchestrator's database is the source of truth for slot ownership, and we stop fighting it.
The sweeper for orphaned claims
The state machine assumes claims become attachments. Real life disagrees. A user closes the tab between clicking generate and the file write completing. A backend crashes mid-claim. A network partition swallows the attach call. The slot sits in claimed, files never arrive, and it is now a zombie — booked, useless, blocking new traffic.
A sweeper handles it. Periodic loop, finding slots in claimed older than sixty seconds with no active session, returning them to the pool via the same atomic-UPDATE shape in reverse. Sixty seconds because legitimate claim-to-attach windows are well under one; anything older is a zombie with overwhelming probability. Same pattern we use for every transient state — claim a state, give it a deadline, let a janitor reap the survivors.
Memory packing (~80 containers per 24GB)
A naive Expo SDK 54 sandbox costs roughly 600MB of RAM, dominated by per-container node_modules. Eight hundred megabytes of duplicate JavaScript modules in every container's memory image is absurd; they are identical bytes.
Forge mounts a shared node_modules layer from the host into each container as a read-only bind. Every sandbox sees the same modules at the same path. The OS page cache loads them once. Per-container memory drops from ~600MB to ~80-100MB, and a 24GB VPS comfortably holds around eighty active containers.
This is the load-bearing trick. Without shared dependencies, the math does not work — the pool would cost more than the product earns. With them, gross margin on a paying user is positive from the first generation.
What we gave up
Pool capacity caps the concurrent-active-preview ceiling. We cannot conjure containers faster than the VPS can boot them; a spike past the controller's headroom shows up as queue depth and, briefly, as cold starts. The mitigations are over-provisioning during predictable peaks, a clear "your preview is warming up" state when the pool is empty, and treating spike events as feedback to the sizing parameters rather than as bugs.
We also accepted that the pool means money sitting idle. Eighty warm sandboxes nobody has touched in the last thirty seconds are eighty containers we are paying for. That is the trade — the alternative is a six-second wait that kills the product. The right answer is to make idle sandboxes cheap, not eliminate them.
This pattern transfers anywhere first-request latency dominates and per-request work is small compared to setup. Warm CI runners. Provisioned-concurrency Lambda. Database connection pools, the original example everyone forgets is the same shape. If you can amortize a long warmup across many short requests, you have a pool problem, and the engineering above is roughly the playbook.
The deeper lesson is that "sub-two-second" is not a performance target you tune toward — it is an architectural decision you commit to at the start, and then everything in the stack either bends to support it or gets removed. Forge's warm pool is the bend. The chat-box-to-phone loop is the consequence.