In May 2026, a company called Subquadratic released SubQ 1M-Preview — the first commercially available LLM built on a fully subquadratic sparse attention architecture rather than the standard transformer. The headline numbers: a 12 million token context window, roughly one-fifth the inference cost of frontier models, and up to 52x faster attention computation at scale.
This isn't just another benchmark announcement. It's worth paying attention to why the underlying architecture matters.
The Quadratic Bottleneck in Standard Transformers
Standard transformer attention has O(n²) complexity relative to the input sequence length. For short sequences this is manageable. For very long ones, it becomes a hard constraint on both cost and feasibility.
# Standard attention complexity
# n = sequence length, d = model dimension
# memory and compute: O(n^2 * d)
# At 100K tokens:
n = 100_000
operations = n ** 2 # 10,000,000,000
# At 1M tokens:
n = 1_000_000
operations = n ** 2 # 1,000,000,000,000,000This is why even frontier models with nominally large context windows often degrade in quality or cost at extreme lengths. The quadratic wall is real.
What Subquadratic Attention Does Differently
Subquadratic attention approaches reduce complexity to something below O(n²) — often O(n log n) or better — through various means: sparse attention patterns, kernel approximations, or state-space models like Mamba. SubQ's specific implementation isn't fully public, but the claim of "fully subquadratic sparse architecture" suggests it's a first-principles design choice rather than a bolt-on approximation.
The 12 million token context window is a natural consequence: once the quadratic bottleneck is gone, extending context becomes much cheaper.
What a 12M Token Context Actually Enables
To put 12 million tokens in perspective:
- A large monorepo with 500K lines of code fits comfortably
- Multi-year document archives can be queried in a single request
- Legal or financial datasets that previously required chunking can be processed end-to-end
The more important question isn't the window size itself, but whether the model accurately attends to relevant tokens across that full range. That requires careful evaluation on your specific workload — not just trusting the headline number.
Cost Implications for Production LLM Applications
At roughly one-fifth the cost of frontier models, SubQ-class pricing could change the economics of LLM-integrated products significantly.
# Rough cost comparison (illustrative)
FRONTIER_COST_PER_M_TOKENS = 15.00 # USD
SUBQ_COST_PER_M_TOKENS = 3.00 # USD
monthly_tokens_m = 100 # 100M tokens/month
frontier_monthly = monthly_tokens_m * FRONTIER_COST_PER_M_TOKENS # $1,500
subq_monthly = monthly_tokens_m * SUBQ_COST_PER_M_TOKENS # $300
annual_savings = (frontier_monthly - subq_monthly) * 12 # $14,400/yearThat said, price per token is only one variable. Reliability, quality on your specific tasks, data residency requirements, and vendor maturity all factor into a production decision.
Our Take
The commercial release of a subquadratic LLM matters because it proves the architecture is viable at scale, not just in research papers. The transformer is no longer the only game in town.
For teams building LLM-powered products: don't rush to adopt SubQ because of the cost numbers, but do start evaluating it seriously if you're processing large volumes of long-context tasks. Run your own evals on representative data before any production migration.
The next 12–18 months will clarify how subquadratic models hold up under real-world conditions. The architectural diversification happening right now is one of the more interesting developments in the LLM space since the original transformer paper.