#Vercel#AI Gateway#LLM#AI SDK#Architecture

Vercel AI Gateway Virtual Models: Swap LLMs Without Redeploying

webhani·

Every team that ships an LLM feature eventually hits the same chore: a new model appears, or an old one gets expensive, and someone has to change a model string in six repositories, review it, and redeploy. Vercel's AI Gateway now offers a way to take that string out of your code: virtual models.

This post covers what the feature is, how we would structure a project around it, and the failure modes worth planning for. Details are based on Vercel's changelog and docs; confirm current plan availability and limits there, since pricing and availability were not stated in the announcement.

What a virtual model is

A virtual model is a saved configuration in AI Gateway: a slug, an underlying model, and the settings requests should use. Your application calls the slug as if it were a model ID (Vercel's example looks like vmc/coding-agent), and the gateway resolves it. You change the underlying model in the dashboard or CLI, and every caller picks up the change without a redeploy.

According to Vercel, this works across model types (language, image, video, speech, embedding, realtime) and through the AI SDK as well as the OpenAI- and Anthropic-compatible APIs. Virtual models that point at language models are also listed in the model catalog the gateway serves to coding agents.

Using it from the AI SDK

From the application side, nothing special is required. The model is just a string:

import { generateText } from 'ai';
 
export async function summarizeTicket(body: string) {
  const { text } = await generateText({
    // A virtual model slug created in AI Gateway, not a provider model ID.
    // The exact slug format comes from your gateway configuration.
    model: process.env.SUMMARY_MODEL ?? 'vmc/ticket-summary',
    prompt: `Summarize this support ticket in three bullet points:\n\n${body}`,
  });
  return text;
}

Keeping the slug in an environment variable is optional but useful: it lets you point staging at a different virtual model than production, so you can trial a candidate model with real traffic shape before promoting it.

Where this pays off

One place to change model choice. If several services call the same capability (ticket summaries, code review comments, classification), a virtual model per capability gives you a single switch. This is the DRY argument, applied to infrastructure configuration.

Faster response to provider changes. Deprecations and price changes arrive on the provider's schedule, not yours. Moving a virtual model to a successor is a configuration edit, not a release.

Cleaner separation of ownership. Application engineers decide what a capability does; whoever owns cost and quality decides which model serves it. That split is hard to enforce when model IDs live in code.

Where it can hurt

A level of indirection is also a level of hidden change. These are the issues we would plan for before adopting it widely.

  1. Silent behaviour changes. Swapping the model behind a slug changes output style, length, refusal behaviour, and tool-calling reliability. Anything that parses model output (JSON, tool arguments) can break without a deploy to correlate it with.
  2. Weaker audit trail. A code change has a pull request. A dashboard change may not. Decide who can edit virtual models, and record changes somewhere your team reads.
  3. Evaluation drift. If your regression suite hardcodes a provider model ID, it no longer tests what production runs. Evaluations should call the same slug production uses, or resolve it first.
  4. A new dependency on the gateway. Your application now needs the gateway to resolve the slug. That was already true if you routed through it, but it is worth stating in your failure analysis.
  5. Cost surprises. A one-click move to a larger model changes spend for every caller at once. Pair changes with budget alerts.

A rollout approach we would use

  1. Start with one low-risk capability, such as internal summarization, not a customer-facing agent.
  2. Name slugs by capability, not by model. ticket-summary survives a model change; gpt-fast does not.
  3. Run your evaluation set against the slug before and after every change, and keep the results with the change record.
  4. Use separate slugs for staging and production, and promote by copying the configuration after evaluation passes.
  5. Restrict edit rights and require a short written reason for each change.
  6. Keep structured-output parsing defensive. Validate model output with a schema (for example, zod) and treat failures as a signal to roll back the slug.
import { z } from 'zod';
 
const Summary = z.object({ bullets: z.array(z.string()).length(3) });
 
export function parseSummary(raw: string) {
  const result = Summary.safeParse(JSON.parse(raw));
  if (!result.success) {
    // Emit a metric here; a spike after a slug change is your rollback trigger.
    throw new Error('Summary output failed schema validation');
  }
  return result.data;
}

Our view

Virtual models are a reasonable default for teams running more than one LLM-backed feature, because they turn model selection into an operational decision. They are less compelling for a single feature owned by a single team, where a constant in the code is simpler and reviewable.

The risk is not technical; it is process. If a model swap needs no deploy, it also needs no review unless you create one. Treat a virtual model change like a production configuration change: evaluated, attributed, and reversible. With that in place, the feature removes real friction.

If you are designing an LLM integration layer and want a second opinion on model routing, evaluation, or cost controls, webhani can help.