For 14 years after AlexNet in 2012, AI = GPU. But in November 2025, NVIDIA announced its own CPU (Vera). AWS and OpenAI signed a $38B deal that explicitly mentioned "tens of millions of CPUs." Intel's CEO declared that "the CPU is the orchestration layer of the AI stack." What changed — and what do the primary sources actually show?
It started with AlexNet winning ImageNet in 2012 (see EP02). Hinton's two students used two NVIDIA GTX 580s — consumer gamer cards. From that day, every AI company started buying NVIDIA GPUs.
The reason this lasted 14 years: AI training is fundamentally matrix multiplication. Each neural network layer = `W × x` matmul. One step of GPT-4 = updating roughly 1.5 trillion weights simultaneously. Every cell computes independently — *embarrassingly parallel*. Just run the same op tens of thousands of times in parallel.
| Item | CPU | GPU |
|---|---|---|
| Cores | Tens (complex cores) | Thousands to tens of thousands (simple cores) |
| Design philosophy | Latency — finish one task fast | Throughput — same task many times in parallel |
| Memory bandwidth | DDR5 ~50 GB/s | HBM3e ~1,500 GB/s (~30×) |
| Specialized circuits | Branch prediction · OoO · large caches | Tensor Cores (FMA · systolic array) |
| AI training speed | baseline | 10-100× (workload-dependent) |
GPUs are weak at "branchy and varied work" — but that doesn't matter for training. That's how an NVIDIA H100 sells for $30,000+ and the company's market cap crossed $4 trillion (see EP06).
As of 2026, roughly 2/3 of AI compute happens in inference — a number cited in Deloitte's 2026 Tech Trends Outlook. In 2023 it was 1/3.
And inside inference, Agentic AI — not single-shot ChatGPT answers but systems like GitHub Copilot Workspace or Claude Computer Use that use tools, search, evaluate results, and retry — became dominant (see EP08 RAG).
One agentic AI cycle has 5 phases:
Of these 5, ②③⑤ are entirely CPU work, and ①④ mix GPU and CPU. Model inference (the GPU's part) is the shortest part of the cycle — not a fixed share.
In November 2025, a joint Georgia Tech and Intel team posted to arXiv — "Towards Understanding, Analyzing, and Optimizing Agentic AI Execution: A CPU-Centric Perspective" (Raj, Kundu, Vohra, Wang, Krishna · arXiv:2511.00739).
They measured 5 real agentic workloads on two systems:
The result — tool processing (CPU work) accounted for 50-90% of end-to-end latency:
| Workload | CPU share | Main CPU work |
|---|---|---|
| RAG (Haystack) | 81-89% | Tool retrieval, vector DB search |
| Toolformer | 77-88% | LLM inference itself (incl. orchestration) |
| Web-Augmented Agent | 48-55% | Summarization (Python heavy) |
| ChemCrow (chemistry) | 85-88% | Conformer generation (molecular) |
| SWE-Agent (coding) | 25-65% | Bash · Python sandbox execution |
On energy too — at large batch sizes, CPU dynamic energy hit 44-61% of total system energy (same paper). It's not just GPUs burning power.
This workload shift shakes the foundations of datacenter design. Market research firm TrendForce (April 2026 report) summarized:
The same direction is confirmed at Intel HQ. Intel CEO Lip-Bu Tan said the following at the Q1 2026 earnings call:
"AI training infrastructure typically runs at a ratio of about 7-8 GPUs per 1 CPU. As we shift to inference, that ratio narrows to about 3-4 GPUs per 1 CPU. As we move to agent and multi-agent environments — that ratio becomes 1:1, or even tilts slightly toward CPUs."
— Lip-Bu Tan, Intel CEO · Q1 2026 earningsAnd one more line from the same call captures the essence:
"CPU is now the orchestration layer and critical control plane for the entire AI stack."
— Lip-Bu Tan, Intel CEO · 2026.04Beyond theory, look at what's actually happening in the market. On November 3, 2025, OpenAI signed a 7-year, $38B deal with AWS. The official joint statement specified the resource scope:
"OpenAI is accessing AWS compute comprising hundreds of thousands of state-of-the-art NVIDIA GPUs, with the ability to expand to tens of millions of CPUs to rapidly scale agentic workloads."
→ Hundreds of thousands of GPUs vs tens of millions of CPUs. The units differ by 100×. A direct signal from OpenAI: scaling agentic workloads needs CPUs in volumes 100× higher than GPUs.
NVIDIA GPUs (GB200/GB300) cluster on Amazon EC2 UltraServers, with CPUs separately specified as agentic-workload infrastructure. All resources are coming online by end of 2026. The signal this single deal sent: agent-era infrastructure = GPU + an overwhelming number of CPUs.
The strongest signal — the absolute champion of the GPU era announcing its own CPU. NVIDIA Vera CPU, announced at GTC 2026, official specs:
| Item | NVIDIA Vera CPU |
|---|---|
| Cores | 88 NVIDIA Olympus cores (custom Armv9.2) |
| Process | TSMC N3, monolithic die + adjacent dielets |
| Multithreading | Spatial Multithreading — 2 tasks per core |
| Memory bandwidth | 1.2 TB/s LPDDR5X · 14 GB/s per core |
| Memory capacity | Up to 1.5 TB |
| GPU link | NVLink-C2C 1.8 TB/s coherent (shared memory with GPU) |
| Notable | Industry first FP8-precision-capable CPU |
| Comparative efficiency | "2× efficiency, 1.5× speed vs prior CPUs" (NVIDIA official) |
| Launch | Mass production starts H2 2026 |
Vera isn't sold standalone. It ships inside the Vera Rubin NVL72 rack with 72 Rubin GPUs + 36 Vera CPUs. NVIDIA's message is clear — "Don't just buy our GPUs. Buy them with our CPUs."
| Product | Cores | Process | Notes |
|---|---|---|---|
| Arm AGI CPU | 136 | TSMC N3 | Launched 2026.03 · Customers: Meta, OpenAI, Cloudflare |
| AMD EPYC Venice | 256 (512 SMT) | TSMC N2 | 2026 launch · 5th-gen Turin successor |
| Intel Xeon 6+ (Clearwater Forest) | 288 | Intel 18A | Delayed to 2027 |
| AWS Graviton5 | 192 | TSMC N3 | Arm Neoverse · AWS in-house |
| Microsoft Cobalt 200 | 132 | TSMC N3 | Azure in-house ARM CPU |
| Google Axion C4A/N4A | 96 / 64 | — | Google Cloud in-house |
Every hyperscaler is now building its own ARM-based datacenter CPU — that's the structural shift. The x86 datacenter monopoly is effectively over. ARM Neoverse is becoming the new standard.
Beyond theory and announcements, the actual Q4 2025 revenue figures confirm the trend.
AMD's CFO noted explicitly that "large enterprises deploying on-prem EPYC more than doubled in 2025." And the 2026 guidance — "server CPU market to grow strong double digits."
2026 AI infrastructure is no longer "just buy GPUs." The chips have differentiated by workload:
1. GPUs aren't dying. GPUs are still the heart of inference and training. There's just a new massive zone *outside* the GPU's domain, and that's where CPUs come back. "GPU does the thinking, CPU does everything else" — and the *everything else* now exceeds half of the workload.
2. NVIDIA building its own CPU (Vera) is the strongest signal. Not portfolio expansion — recognition that in the agent era, a weak CPU next to your GPU drags down GPU revenue too. CPU+GPU integrated packages with shared memory via NVLink-C2C are becoming standard.
3. AMD is the largest near-term beneficiary — EPYC is effectively the agent-era standard CPU. Q4 2025 datacenter +39% YoY, EPYC Turin 50%+ share, 2026 guidance strong double digits. Intel faces a near-term crisis from production delays — demand exploding, supply short.
4. ARM's datacenter penetration is decisive. Vera, AGI, Graviton5, Cobalt 200, Axion — all ARM Neoverse. The x86 datacenter monopoly is effectively over. AMD x86 stays strong near term, but on a 5-year horizon ARM could take 30%+ share.
5. Memory bandwidth is the new battlefield. Vera's headline isn't 88 cores — it's 1.2 TB/s LPDDR5X + 1.8 TB/s NVLink-C2C. Competition shifts from core count to memory bandwidth (HBM4, LPDDR5X, CXL 3.0). Beneficiaries — SK hynix, Micron, Samsung memory.
The 2012-2024 era of AI was the "matmul era" — GPUs won decisively. The 2025-2030 era is the "agent orchestration era" — *heterogeneous compute* of CPU + GPU + NPU + memory becomes the new default architecture.
2025-2026 — when NVIDIA built its own CPU, when AWS wrote "tens of millions of CPUs" into an OpenAI contract, and when Intel's CEO declared "CPU is the orchestration layer" — was the inflection point.
The CPU's revival isn't a GPU defeat — it's the consequence of agents emerging as a new workload category. AI infrastructure decisions in 2026 are shifting from "how many GPUs do we buy" to "how do we balance CPU, GPU, memory, and interconnect."