What 80 user app containers on a single 24GB VPS taught us about memory packing

Naive packing = 12 containers per VPS, swap-bound. Tuned packing = ~80 containers, 80-100MB each. Four wins, one ceiling we hit, and the suspend pattern we wish we'd built earlier.

AppX team · 2026-05-21

Eighty user app containers on a single 24GB VPS. That is where we ended up. It sounds like a stunt number until you remember each container is running a full Metro dev server plus a freshly generated React Native codebase, and each one belongs to a different person actively iterating on their app. Eighty live previews. One box. The math only works because every container averages 80-100MB of resident memory, and that number is the output of a year of memory engineering, not a default you get from docker run.

This post is what we learned getting there.

The naive packing

The first version of AppX's preview infrastructure did the obvious thing. A user clicks generate, we spawn a container, the container runs npm install into its own node_modules, boots its own Metro instance, and holds its own bundle cache. Clean, isolated, easy to reason about. Roughly 600MB of resident memory per container, dominated by an Expo SDK 54 node_modules tree that weighs ~350MB on disk and then again in Node's module cache once Metro hydrates the graph.

Twelve containers per 24GB VPS before the box started swapping. Sometimes ten if a couple of users were actively bundling. The OOM killer would occasionally pick off whichever Metro process had the worst luck. The economics did not work — at twelve concurrent previews per VPS, the cost per active user was higher than the price of the product.

We had a choice. Buy bigger boxes, or stop wasting memory.

Win 1: shared node_modules

The first realization is embarrassing in retrospect. Every preview container was loading the same bytes. Every user's app pinned the same Expo SDK 54. Same React Native, same React, same lucide-react-native icons. The handful of files that varied per user were the generated screens. Everything underneath was identical across the entire fleet, and we were paying for it eighty times in RAM and eighty times on disk.

So we stopped. The host now keeps one canonical node_modules at /opt/expo-shared-deps/node_modules, and every container gets it as a read-only bind mount. Same path inside the container, same files, same inodes. The Linux page cache loads each module once. Every Metro instance reading react-native/Libraries/Core/InitializeCore.js is reading the same physical pages.

Per-container resident memory dropped roughly 60%. The number that mattered was that the VPS went from holding twelve previews to holding closer to forty without breaking a sweat. The economics started to bend.

The lesson generalizes: every byte you write into a container that already exists somewhere else on the host is a byte you are paying for twice. Find them. They are everywhere.

Win 2: persistent Metro

The intuition we started with was that holding a Metro process alive between requests was wasteful. Cold-spawn Metro per bundle, let it die, free the memory. Defensive against memory leaks, cheap in idle cost.

The intuition was wrong, and measuring it was the only way to see why.

Persistent Metro keeps the module graph in memory forever — that sounds expensive, and the resident-set screenshot at a single point in time agrees. But cold-spawn Metro re-parses the graph, re-resolves every import, re-transforms every TSX file on every request. The CPU spikes were obvious; the hidden cost was the allocation churn. Cold-spawn Metro spent half its life in V8's young generation, constantly allocating and freeing the same parsed-module objects. Persistent Metro allocates them once and holds them, which means a much smaller working set in steady state and far less GC pressure.

Net memory was lower with persistent Metro, not higher. The integral of memory-over-time across a user's session beat the cold-spawn shape by a comfortable margin, and the latency story was not close — every code push hit a hot bundler instead of a paying-for-warmup bundler. We wrote about the latency side of that decision separately in persistent-metro-1750x-faster-code-push. The memory side is what unlocked the packing density.

Win 3: idle eviction

Eighty warm containers is the capacity of the box. It is not the steady-state population. A real user opens AppX, generates an app, iterates for ten minutes, then closes the tab and goes to dinner. The container they were using is now an 80MB ghost holding a slot.

So we put it to sleep. Any container that has not received traffic in thirty minutes gets a polite shutdown — Metro flushes, the process exits, the container is destroyed, the slot returns to the pool. The user's source files are checkpointed to disk so the next time they come back, we can spin up a fresh sandbox and rehydrate. Cold resume costs the user a few seconds; ten minutes ago they had walked away from their laptop, so they do not notice.

The thirty-minute number is empirical. We tried fifteen and it punished users who were thinking. We tried sixty and the pool clogged with abandoned sessions. Thirty is the elbow on the histogram between "stepped away to refill coffee" and "closed the tab and forgot."

Idle eviction is the difference between a system that runs out of slots in a workday and a system that recycles them all day long. The dormant-tab cost is not a small effect — most active users are not actively typing at any given second.

Win 4: load-aware pool sizing

Even with eviction, the cost of keeping warm slots around at 3am is real. A pool of eighty warm sandboxes sitting idle is eighty containers' worth of RAM you are paying for to serve a claim rate of one a minute.

Forge sizes its warm pool from recent demand. A controller loop watches claim rate over a short window, looks at how many idle warm slots are sitting around, and computes a target. Demand goes up, pool grows. Demand drops, the oldest warm slots age out. We wrote about the pool controller's atomic-claim and reconciler shape in forge-deep-dive-warm-pool — the part relevant here is that the pool adapts, which means the average resident-memory cost of the VPS over a day is dramatically lower than its peak.

You stop paying for the night.

The ceiling we hit

At around eighty containers, we started seeing problems that were not about RAM at all.

The Linux kernel maintains a non-trivial amount of per-container bookkeeping. File descriptor tables. Network namespaces. cgroup hierarchies. Each container is cheap, but eighty of them is eighty network namespaces, eighty cgroup branches, several thousand file descriptors, and a kernel slab allocator that starts looking grouchy. We pushed the pool toward 120 and the OOM killer began picking off Metro processes — not because they were bloated, but because kernel memory plus userland memory plus page cache had collectively crossed a threshold. The kernel does not page itself out.

We backed off. Eighty is comfortably below the ceiling, and the ceiling exists for structural reasons we cannot tune away with a config flag. Alpine base images help — the OS layer is small — but Alpine is still Linux, and Linux still wants its slab.

What we'd do differently

The single move we should have made earlier was idle-suspend with on-demand resume. We spent months optimizing the warm-pool model — making each warm slot cheaper, packing them denser, sizing the fleet smarter — when the bigger lever was that most slots should not be warm at all. A pool is great when usage is sustained. A suspend-and-resume system is what kills the cost of the long tail of dormant tabs.

If we were starting today, we would still build the pool, because the chat-to-phone latency requirement does not bend. But we would build idle-suspend the same week and treat warm slots as the minority state, not the default. The warm pool is for active iteration. Everything else is checkpointed and resumed.

Closing

Dense packing on a single VPS is mostly an exercise in finding duplication and removing it. Shared dependencies, shared bundlers, shared base images, shared kernel page cache for shared files. Every byte that exists in two places is one byte more than you needed.

The ceiling is real, but it is structurally above what a builder product needs on a single box. Eighty active previews on 24GB of RAM is not a stunt — it is what falls out when you take the duplication out and refuse to put it back.

Try your own app idea

Describe your app in AppX →