Another model, another evaluation cycle
xAI shipped Grok 4.5 this week, positioned explicitly around coding and agentic tasks rather than as a general-purpose chat model. It's available through the xAI API, in xAI's own Grok Build agent product, and through Cursor on all plans — reportedly trained in collaboration with the Cursor team, which explains the tight integration at launch rather than a bolt-on provider addition weeks later.
For teams that have already standardized on a coding assistant, the temptation is to skim the announcement and move on. We think that's the wrong instinct. Not because Grok 4.5 necessarily belongs in your stack, but because the right response to every new frontier model release is the same lightweight evaluation, not a shrug or a full re-platforming.
What's actually new here
Three details in this release are worth separating from the general marketing noise:
Pricing. Grok 4.5 launched at $2 per million input tokens and $6 per million output tokens — aggressive relative to comparable frontier coding models, and notably asymmetric in a way that rewards workflows with long context and short completions, which describes most agentic coding tasks (large codebase context, targeted diffs as output).
Throughput. xAI reports roughly 80 tokens per second at default settings, with configurable reasoning effort (low, medium, high). For interactive coding sessions, generation speed affects perceived responsiveness as much as raw capability does — a slower model with a better benchmark score can still feel worse to use.
Distribution. Shipping inside Cursor at launch, rather than as a third-party model you manually configure, matters more than it looks. It signals that model providers are increasingly optimizing for "available inside the tool developers already have open" over "available via API for you to wire up yourself."
How we evaluate a new model release
We don't adopt a new frontier model into client work based on launch benchmarks alone — those are provider-selected and rarely reflect your actual codebase or task mix. Our evaluation checklist looks like this:
- Run it against your own repo, not a synthetic benchmark. Pick 3-5 real tickets from your backlog — a bug fix, a small feature, a refactor — and run the same tickets through your current model and the new one.
- Measure tokens-to-working-diff, not just pass/fail. A model that gets to a correct solution in fewer iterations is worth more than raw accuracy suggests, because iteration cost compounds in agentic workflows.
- Check tool-use reliability separately from reasoning quality. Agentic coding depends heavily on a model correctly calling file-read, file-write, and shell-execution tools in sequence. This is a distinct capability from code quality and needs its own test pass.
- Price it against your actual usage pattern, not the headline rate. Input-heavy agentic sessions (a model re-reading a large context window on every turn) behave very differently under the same pricing table than short, isolated completions.
// A minimal harness for comparing models on your own tickets,
// not a public benchmark
interface EvalTicket {
id: string;
prompt: string;
repoSnapshot: string;
expectedDiffPattern: RegExp;
}
async function evaluateModel(
model: string,
tickets: EvalTicket[]
): Promise<{ id: string; iterations: number; passed: boolean }[]> {
const results = [];
for (const ticket of tickets) {
const { diff, iterations } = await runAgenticSession(model, ticket);
results.push({
id: ticket.id,
iterations,
passed: ticket.expectedDiffPattern.test(diff),
});
}
return results;
}The multi-model reality most teams are already living in
Very few teams we work with run a single model exclusively anymore. The typical pattern is a primary model for day-to-day agentic coding, with routing to a second or third model for specific task types — cost-sensitive bulk work routed to a cheaper model, complex architectural reasoning routed to whichever model currently leads on that axis. Grok 4.5's pricing and Cursor-native availability make it a plausible candidate for the "cost-sensitive, high-volume" slot in that kind of setup, without necessarily displacing whatever handles your hardest reasoning tasks.
This is also why provider lock-in is worth actively avoiding at the integration layer. If your agentic coding setup is wired directly to one vendor's SDK, evaluating a new entrant like Grok 4.5 means rewriting integration code before you've even tested whether it's worth switching to. Routing through a model-agnostic interface — whether that's a gateway abstraction or simply consistent prompt/tool-call formatting across providers — keeps the cost of evaluation low.
Our recommendations
- Budget a half-day per quarter for new model evaluation, using the same fixed ticket set each time so results are comparable across releases, not just against the provider's own benchmark claims.
- Don't chase every release. Evaluate meaningfully differentiated launches (new provider, notable pricing shift, new tool-use capability) rather than every point release.
- Separate "is it good at coding" from "is it good at agentic tool use." They're correlated but not identical, and agentic workflows fail more often on the second axis than the first.
- Keep your integration layer model-agnostic so evaluation cost stays low regardless of how fast the frontier model landscape keeps moving.
Takeaways
Grok 4.5 is a legitimate new entrant, not a marketing exercise — competitive pricing, Cursor-native distribution, and a coding-first design all matter in practice. But the more durable lesson from watching this space release after release is that the individual model matters less than your team's ability to cheaply and repeatedly evaluate whichever model is newest. Build that muscle once, and every future Grok, GPT, or Claude release becomes a half-day decision instead of a debate.