Why we moved from direct provider SDKs to OpenRouter
Three SDK clients, three streaming parsers, three error shapes. We switched to OpenRouter in 48 hours. The dividends — and the honest tradeoffs we're paying for.
AppX team ·
Every feature in AppX that matters — generation, edit, chat, classification, summarization — is an LLM call. For a long stretch we shipped those calls against the providers' official SDKs: Anthropic's TypeScript client for one task, Google's GenAI client for another, OpenAI's client for a third. Each integration looked tidy on the day it landed. In aggregate, the codebase carried three different shapes of the same conceptual thing — "ask a model, stream tokens, handle errors, retry on rate limits, count tokens." That aggregate is what eventually pushed us off direct SDKs and onto OpenRouter.
What we left behind
flowchart LR
subgraph before["Before · three direct SDKs"]
A1["App"] --> P1["Provider A SDK"]
A1 --> P2["Provider B SDK"]
A1 --> P3["Provider C SDK"]
end
subgraph after["After · one gateway"]
A2["App"] --> R["OpenRouter"] --> M["any model"]
end
before --> after
style after fill:#13241a,stroke:#2ecc71,color:#d7f7e3
Three SDK clients, each with its own streaming parser. Anthropic's events are content_block_delta blobs. OpenAI's events are choices[0].delta.content chunks. Google's events are a different envelope again. Each one had to be normalized into our internal TokenStream interface before the rest of the system could use it.
Three error class hierarchies. Anthropic.APIError versus OpenAI.APIError versus Google's GoogleGenerativeAIError. The semantics overlap — rate limit, context-length, server error, bad request — but the field names, retry hints, and HTTP status mappings do not. We wrote a normalizer. The normalizer kept growing edge cases.
Three rate-limit handlers. Anthropic returns retry-after hints in one header; OpenAI uses a different convention; Google leans on RPC codes. Backoff logic forked per provider.
Three model-id constants files, three config blocks, three sets of env vars, three billing dashboards. To answer "is this model getting more expensive this month?" we opened three browser tabs.
Total damage: roughly 600 lines of "we should consolidate this someday" code. Plumbing that did not generate a single line of React Native for a user. Every "let's try a new model" started with "okay, which SDK does that provider use, and do we have a shim for it?"
The switch (48 hours)
We had been writing OpenRouter on the someday-list for months. The day we ran out of patience went like this. Morning: created an OpenRouter account, generated a key, read the docs. The endpoint is OpenAI-compatible — same chat-completions shape, same streaming protocol. Afternoon: pointed the openai SDK at OpenRouter's base URL with our key, sent a test call to a Gemini model through it. It returned. The streaming worked. The error shape was the same shape we already handled for OpenAI.
The rest was deletion. Day one: replaced Anthropic and Google SDK clients with the OpenRouter-pointed openai client behind our existing model registry. Day two: deleted the three streaming parsers and collapsed them into one. Deleted two error normalizers. Deleted two rate-limit handlers. Shipped behind a feature flag, ran every generation/edit/chat path through it on staging, flipped to prod.
Forty-eight hours from "let's try this" to "this is now how AppX talks to models." Net diff: about 600 lines removed, about 80 added.
Model-id discipline
OpenRouter exposes models as stable string identifiers — google/gemini-3.1-pro-preview, anthropic/claude-opus-4-7, and so on — with the provider namespace baked into the name. We do not hardcode these in service code. They live in our ai_models table with a tier (pro / flash), a default flag, and a credit cost.
The discipline matters more than it sounds. Every service that needs a model asks the registry by tier ("give me the architect model") and gets back the current ID plus its cost. Swapping from anthropic/claude-opus-4-7 to google/gemini-3.1-pro-preview for the architect tier is one row update — no deploy, no code change, no SDK install. The tier abstraction outlived a Gemini version bump and a Claude version bump without either touching application code.
Fallback patterns
This is the underrated win. OpenRouter accepts a fallback array on each request — primary model first, alternates after. If the primary's upstream is degraded, the router transparently retries on the next. The response still comes back in the same shape.
We use it on both AI tiers. The architect tier (the planning model that decides what the edit looks like) has a Pro fallback chain — if our primary Pro hiccups, the router rolls to a sibling Pro from another provider. Same for the editor tier, which leans on Flash-class models for cost reasons. Both chains are configured per environment, not in code.
Doing this against three SDKs directly would have meant building a cross-provider retry layer ourselves, with its own error taxonomy, its own observability, its own bugs. The router gave it to us for free.
Honest tradeoffs
One more dependency. If OpenRouter is down, every LLM call in AppX is down. In about six months of running through it we have hit two short incidents; both resolved before they became customer-visible at our scale. If we ran a higher tier of business we would be thinking harder about a direct-SDK fallback for the worst case.
Router markup. OpenRouter charges a small single-digit percent over provider list price. Compared to the engineering time we were spending on three SDK shims, the markup is not close to break-even — we are paying for velocity and we are getting velocity. At dramatically larger volume the math might shift.
Less control over provider-specific features. The router exposes the common subset cleanly, and surfaces most major features (tool use, structured output, streaming) consistently across providers. But bleeding-edge stuff — Anthropic's prompt caching when it first shipped, OpenAI's structured-output mode the week of release — sometimes lags behind direct-SDK availability. If your product depends on a feature the router has not surfaced yet, you have to wait or fall back to the direct SDK for that one path.
When NOT to switch
If you are using exactly one provider, do not introduce a router. The benefit of OpenRouter is multi-provider normalization, and you do not have multi-provider.
If a feature you genuinely need is direct-SDK-only — aggressive prompt caching against one vendor, for instance, where the cache-hit economics drive your unit cost — staying on the direct SDK for that path is correct. The router is excellent at the 90% shared surface; the 10% edges are real.
If your product is small enough that you will never call more than one model, the consolidation gain does not exist.
For everyone else with two or more providers in flight, the engineering bill on direct-SDK consolidation is the kind of cost that compounds quietly until you notice it owns a meaningful slice of your repo.
Closing
The frame we settled on: OpenRouter is plumbing. The same way Forge is plumbing for compute, OpenRouter is plumbing for model calls. The interesting work at AppX is the AI generation pipeline, the chat UX, the prompts that turn "make me a habit tracker" into a working app. Anything that lets us spend less brain on model-vendor mechanics and more on the prompts and the product is a good trade. The router cleared an entire category of plumbing off our plate in two days. Some of the best decisions look like that in retrospect — small, fast, and embarrassingly overdue.