The Inference Inflection — Why the CPU Shortage Is Getting Real

Over the past few days, remarks from key figures across the AI industry have thrown one clear trend into sharp relief: **the age of inference has well and truly arrived**.

OpenAI's Sam Altman said, "We now have to become an AI inference company," and Noam Brown noted that "inference compute is a strategic resource and is currently underestimated." NVIDIA's Jensen Huang likewise declared in his GTC keynote that "the inflection point of inference has arrived," claiming that compute demand has grown **a million-fold over the past two years**.

What deserves attention is not just GPUs but the **surging demand on the CPU side**. Intel CEO Lip-Bu Tan explained on the Q1 earnings call that production deployments of agents and RL training environments are consuming massive amounts of CPU resources. SemiAnalysis's analysis likewise points out that although the COVID-era server refresh cycle has arrived, companies have been directing their budgets toward GPUs — and as a result, **a CPU shortage is starting to bite**.

Prefill/decode disaggregation is becoming standardized, and with NVIDIA's acquisition of Groq, the Intel–SambaNova partnership, and Amazon's tie-up with Cerebras, the wave of inference optimization is spreading rapidly.

OpenAI Codex — From Coding Tool to General-Purpose Work Surface

OpenAI's **Codex** is evolving from a mere coding assistant into a **general-purpose work environment**.

Concretely, its use is expanding into non-coding knowledge work such as synthesizing research, manipulating spreadsheets, and tracking decision-making. For Business/Enterprise customers, **dedicated Codex seats (at $0 per seat)** are being offered through the end of June, and a Supabase integration and a Figma plugin (which turns implementation plans into FigJam boards) have also been added.

On the performance front, moving the Responses API to **WebSocket mode** keeps state across tool calls, delivering **agent workflows that are up to 40% faster**. On the VS Code side, improvements to the harness (the execution environment for agents) — such as semantic indexing, cross-repo search, and prompt evaluation extensions — continue apace.

The key point is that the recognition — that **it is not just model capability but harness quality that determines agent outcomes** — is spreading across the industry.

The Cursor SDK and the Platformization of Agent Harnesses

Cursor has released the **Cursor SDK**, which makes Cursor's internal runtime, harness, and models usable in CI/CD pipelines, automation, and product-embedded agents.

This is more than just an API release — it is worth noting as a move to reposition Cursor from an "IDE product" into a **programmable agent foundation**. Taken together with OpenAI Codex turning into an app server and VS Code's harness improvements, the coding-agent industry is clearly converging on **headless agent runtimes plus usage-based pricing**.

Meanwhile, LangChain has introduced **Harness Profiles** in its **Deep Agents** product family, making it possible to version-control per-model prompts, tools, and middleware. The company emphasizes the importance of open harnesses and open evaluations — a move that responds to the rising costs of closed models.

Cloudflare, for its part, has built a mechanism that lets agents themselves become **Cloudflare customers** (creating accounts, registering domains, and starting paid plans). A trend is emerging that treats agents not merely as helper tools but as first-class actors in business workflows.

Qwen's FlashQLA and vLLM's Blackwell Optimization — Inference Kernel Competition Heats Up

Alibaba's Qwen team has released **FlashQLA**, a high-performance linear attention kernel built on TileLang that delivers 2–3x speedups, especially for **long-context workloads on small models**. Its design combines gate-driven automatic in-card CP, algebraic reformulation, and fused warp-specialized kernels, and it explicitly targets **agentic AI on personal devices**.

vLLM has also achieved an industry-leading speed of **230 tok/s for DeepSeek V3.2** on Blackwell (HGX B300). This is the result of combining NVFP4 quantization, EAGLE3+MTP speculative decoding, and per-model kernel fusion. SemiAnalysis also reports that vLLM 0.20.0 and the MegaMoE kernels are delivering major results for DeepSeek v4 Pro on GB200.

Zhipu AI, meanwhile, has published a post-mortem on **GLM-5** serving, reporting fixes for a KV cache race condition and a HiCache synchronization bug, as well as up to 132% throughput improvement for long-context coding-agent serving thanks to **LayerSplit**.

An Empirical Analysis of AI Models' "Japan Bias"

According to research by a European team reported by ITmedia, it has been empirically demonstrated that the outputs of some AI models (such as GPT-4o-mini) **skew strikingly toward Japanese culture**.

The study confirmed a phenomenon in which, no matter which country or culture a question is about, the answers get pulled back into a Japanese context, with biases in training data and the influence of fine-tuning cited as likely causes. This research is drawing attention for raising a fundamental question: the "neutrality of AI."

---