How the AI pipeline evolved: single-shot to plan-then-build to architect+editor
Three architectures in eight months. Each pivot was triggered by failures the prior architecture couldn't fix without adding the next layer. The shape we landed on, and what we think Era 4 looks like.
AppX team ·
An AI-powered app builder is, at its core, a function: natural language in, working code out. The shape of that function looks simple from the outside. Inside, it is a pipeline, and the pipeline has architecture, and the architecture decides what kind of products you can ship.
At AppX we have rewritten that pipeline three times in eight months. Each rewrite was a structural pivot, not a tuning pass. Each was triggered by a class of failures the prior architecture could not fix by adding more retries, more validators, or more tokens. This is the story of how we got from one LLM call to two specialized loops, and what we think comes next.
Era 1: Single-shot generation
flowchart LR
P["Your prompt"] --> L["One big LLM call"]
L --> A["The whole app, in one shot"]
A -.->|"one bad file = whole app broken"| X(("brittle"))
style X fill:#3a1414,stroke:#c0392b,color:#f7d7d7
The first version was the simplest thing that demoed. One model call. The prompt carried the user's description, a system preamble, and a JSON schema. The output was a map of file paths to file contents. Parse the JSON, write the files to disk, hand them to the bundler. Done.
It worked beautifully for a single-screen weather app. It worked for a two-screen todo list. It worked for the demo we showed investors.
It collapsed somewhere around three screens with shared state.
The failure modes were not bugs — they were properties of the architecture. Once a project crossed a few files, the model could no longer hold the whole thing in working memory while it was emitting tokens. It would import Button from ./components/Button in one file and ./ui/Button in another. It would define a ThemeColors type in one file and silently redefine an incompatible version in the next. It would pick Home as a route name in the navigator and HomeScreen everywhere else. Each file looked correct in isolation. The project as a whole would not compile.
Worse, there was no notion of editing. The user types "make the primary button blue." The model regenerates the entire project. Eight files come back. One of them has a different button color. Seven of them are subtly different from before — a hook order changed, a prop renamed, an import reshuffled. The new project compiles or it doesn't, and either way the user has lost everything they iterated on.
Token cost was the other axis. Every regeneration paid the full output cost of the entire project. Cost scaled with project size times session length. We watched the unit economics on a real session and stopped showing the numbers in meetings.
The architectural ceiling was clear: a single LLM call cannot maintain cross-file invariants in any project large enough to be interesting. The model has no shared contract between the file it emitted thirty seconds ago and the file it is emitting now. There was no clever prompt that fixed this, because the prompt was not the problem. The shape of the call was the problem.
Era 2: Plan-then-build
flowchart LR
P["Your prompt"] --> PL["Plan<br/><i>fast model</i>"]
PL --> G["Generate files<br/><i>strong model</i>"]
G --> V["Validate"]
V -.->|"still regenerates everything on a miss"| G
style PL fill:#13203a,stroke:#5e6ad2,color:#dbe2ff
The pivot was to split the work into two phases with a structured artifact between them.
A planner model — a Pro-tier reasoning model — read the user's intent and produced a GenerationPlan. The plan named every file the project needed, the dependency graph between them, the component hierarchy, the data flow, the route table. Then a cheaper, faster builder model — Flash tier — generated each file in topological order, with the plan as part of the prompt for every file.
The plan looked something like this:
type GenerationPlan = {
routes: { name: string; screen: string; params?: string[] }[];
files: {
path: string;
role: 'screen' | 'component' | 'hook' | 'type' | 'util';
exports: { name: string; kind: 'fn' | 'type' | 'component' }[];
imports: { from: string; names: string[] }[];
}[];
theme: { primary: string; surface: string; text: string; /* ... */ };
dataModel: { name: string; fields: { name: string; type: string }[] }[];
};
The plan was the shared contract. The builder model could not invent a new theme color, because the theme was in the plan. It could not import from a file that did not exist, because the file list was in the plan. It could not pick a different route name in the navigator versus the link, because route names were in the plan.
Before any tokens were spent on generation, we ran Kahn's topological sort on the dependency graph. Cycles got caught at the planner level — much cheaper to ask the planner to re-emit a plan than to discover halfway through generation that file A imports file B which imports file A. Generation order itself fell out of the sort: types first, hooks next, components, screens, then the router.
Coherence improved by a lot. Cost dropped — most of the output tokens were now coming from the cheap model. Generation became somewhat parallelizable; independent leaves of the dependency graph could build at the same time.
The ceiling this hit was iteration. The plan-then-build pipeline was still a generator. It generated whole projects. It did not edit them. When a user said "make the button blue," we either regenerated everything (back to Era 1 economics) or we wrote increasingly elaborate scaffolding to detect "small edits" and route them somewhere else. The "somewhere else" was a hack — a separate prompt, a separate model, no shared contract with the planner, no way to evolve the plan as the project evolved.
We had built a project bootstrapper. We had not built an editor. Users wanted both.
Era 3: Architect + Editor
flowchart LR
P["Your prompt"] --> AR["Architect<br/>plans the change"]
AR --> ED["Editor<br/>applies SEARCH/REPLACE diffs"]
ED --> V["Validate"]
V -->|"pass"| DONE["Ship"]
V -.->|"fail · re-plan once with context"| AR
style AR fill:#13203a,stroke:#5e6ad2,color:#dbe2ff
style DONE fill:#13241a,stroke:#2ecc71,color:#d7f7e3
The third pivot was to stop treating "generate" and "edit" as variations of the same call. They are different jobs. They want different models, different prompts, different tools, different cost profiles.
We split the pipeline into two loops that own different parts of a project's lifecycle.
The Architect is a Pro-tier model — Gemini 3.1 Pro in our case. It owns structural decisions. It runs when the project is new, or when an edit is large enough to need a fresh plan. It calls submit_plan and emits something close to the Era-2 GenerationPlan, evolved to describe deltas rather than only greenfields.
The Editor is a Flash-tier model. It owns surgical edits expressed as SEARCH/REPLACE blocks against specific files. It does not invent structure. It does not pick route names. It applies a contract.
The tool-call flow looks roughly like this:
// architect turn
architect.run({
tools: ['read_file', 'list_project_files', 'submit_plan', 'ask_user'],
});
// -> submit_plan({ intent: 'modify', files: ['screens/Home.tsx'], summary: '...' })
// editor turn, scoped to the files the architect named
for (const file of plan.files) {
editor.run({
tools: ['read_file', 'apply_edit'],
// apply_edit takes diff-fenced SEARCH/REPLACE blocks
// matched in three tiers: exact -> whitespace-normalized -> line-range
});
}
Two things this unlocked. First, iteration economics. A user can land fifty edits in a session — color tweaks, copy changes, new fields on a form, a new tab in the navigator — and the Pro model only runs when the plan itself needs to change. The Flash model handles the long tail. Cost-per-session dropped by an order of magnitude on iteration-heavy traffic.
Second, the read-before-edit invariant became enforceable. Every apply_edit checks that the editor has actually read the current contents of the file it is patching (we hash the file and compare). This catches an entire class of "the model is patching a stale mental model of the file" failures that used to surface as silent corruption. If the hash doesn't match, the architect re-plans with the failure context. One retry. No infinite repair loops.
The SEARCH/REPLACE protocol earns its keep here. A unified diff is too brittle (line numbers drift). A whole-file rewrite is too expensive and too prone to drift the parts you didn't want changed. SEARCH/REPLACE with the three-tier matcher — exact, whitespace-normalized, line-range — handles the realistic spectrum of edits at the cost of a small validator on top.
What stayed across all three eras
Three things did not change.
The validation pipeline always ran last. Syntax check, web-element check (no <div> in React Native), import resolution, lucide icon resolution, theme token resolution. Era 1, Era 2, Era 3 — every output goes through the same validators before it touches a user's preview. The pipeline got smarter; the gate stayed.
Cost metering was always upstream of generation. Credits get reserved before the call, deducted on success, refunded on failure. This survived three rewrites because it is a property of the surrounding system, not the AI loop itself.
And the preview loop never changed. Code lands on disk, the Forge container picks up the change, the bundler rebuilds, the user's phone reloads. The AI pipeline above it could be one model, two models, or twenty models. The shipping surface stayed the same.
What we think Era 4 looks like
Honest answer: we do not know yet. We have hypotheses, and we are watching for the failure mode that will force the next pivot.
The candidate we keep returning to is the architect should be able to run code. Today the Architect plans and the Editor patches; neither runs the project to see what happens. A class of subtle bugs — runtime errors, layout regressions, navigation flows that compile but break at the second tap — would be cheaper to catch by running than by predicting. A pipeline where the Architect can install a dep, run a build, read a log, and choose to re-plan based on real runtime evidence is the obvious next step.
The risk is cost and latency. A generation that runs your project is slower and more expensive than one that doesn't. We are watching for the threshold where the cost of one more "model writes code blindly" failure exceeds the cost of one more "model verifies its work" success. Bloom-style products are already past it. We are not, yet. We will be.
The other candidate is multi-agent planning — splitting the Architect itself into role-specialized planners (data model planner, navigation planner, theme planner) that negotiate. We are skeptical. Every multi-agent system we have built has gotten worse before it got better, and most stopped at worse.
Closing
The honest summary is that every era of the pipeline was correct for the products we were trying to ship at the time, and wrong for the products we wanted to ship next. Single-shot was right for "demo this idea." Plan-then-build was right for "make new things coherent." Architect+Editor was right for "let people live in their projects."
The mistake we caught ourselves making, twice, was trying to patch the current era past its ceiling instead of admitting the ceiling was the architecture. Every time we did that we lost a month to scaffolding that we deleted in the next pivot anyway. The faster move was always: name the class of failure the current pipeline cannot fix without becoming the next pipeline, and just become it.
We expect to throw away Era 3 too. Probably sooner than is comfortable. That is the job.