This is a briefing memo for the Curious Mind community. It reflects my personal views on a fast-moving topic and is not a research report.
A friend of mine runs an autonomous coding agent on his laptop. Last week he watched it work for eleven minutes on a single task. It searched his codebase, wrote three functions, ran them, found a bug, read the stack trace, rewrote the failing function, ran it again, passed the tests, opened a pull request, and went quiet.
For eleven minutes, the GPU in his machine sat mostly idle.
The CPU did almost all the work.
For three years, the tech-finance consensus has been that the world runs on GPUs. Nvidia’s market cap tells you the same story. So do the headlines, the venture rounds, and the cover stories from publications that had never cared about semiconductors before 2023. The GPU was the hero of the training era, the engine that took a trillion tokens of text and compressed them into something that looked like thought.
But the training era was the first act. The second act, the one that started quietly in 2025 and picked up speed through this year — is not about building models. It’s about using them. And when you use an AI model the way a person actually uses a brain — pausing, checking, reconsidering, looping back — the hardware that matters changes.
The CPU, the boring brain at the center of every server, is having a second life. Not a revival of nostalgia. An actual, structural, unavoidable comeback. And very few portfolios are positioned for it.
The Hollowing
Remember how the CPU got sidelined.
Before 2012, the central processor was the whole show. It ran your operating system, your database, your spreadsheet, your browser. It was built for what engineers call scalar math, logic that walks through problems one step at a time. If this, then that. When deep learning started working in 2012, the industry discovered that neural networks don’t think this way. They multiply massive matrices of numbers in parallel, billions of identical little operations at once, and they don’t care about branching or order.
The GPU, originally designed to push pixels onto a screen, was accidentally perfect for this. From 2018 to 2024, the data center reorganized itself around it. The CPU became the traffic cop — the guy at the factory door feeding raw material to the assembly line. The real work happened elsewhere.
The hollowing was brutal. Intel lost half its market cap. AMD’s data center business became a sideshow next to its consumer chips. ARM was treated as a mobile story. Everyone was either building GPUs, buying GPUs, or losing money for not having bought enough GPUs.
And then the nature of the work changed.
System 1 and System 2
Kahneman’s framework for human cognition maps surprisingly well onto this shift. System 1 is fast, automatic, pattern-based. System 2 is slow, deliberate, logical.
The first versions of ChatGPT were pure System 1. You typed a question, the model predicted the next word, and the next, and the next, in one unbroken rush, no pausing, no checking, no looping back. That kind of work is a GPU’s dream: one massive, parallel forward pass.
The models of 2026 are different. They plan, use tools, search the web, write code, run it, read the output, decide what to do next. They pause for seconds or minutes. They fail, notice the failure, and try again. This is System 2 thinking, and it looks almost nothing like matrix multiplication. It looks like a person working through a problem.
The more capable AI becomes, the less of its time is spent doing pure forward-pass inference. A reasoning agent running for eleven minutes might spend thirty seconds on GPU-heavy generation and ten minutes and thirty seconds on CPU-heavy orchestration — deciding what to do next, calling tools, reading results, managing memory, walking sequentially through logic.
GPUs are bad at this. They are phenomenal at doing the same operation across ten thousand cores. They are terrible at making ten thousand different decisions. Every time an agent branches, if the test fails, do this, otherwise, do that, the GPU’s parallel cores mostly sit idle while one thread decides. The whole machine pauses for one instruction.
CPUs were born for this work. They have fewer cores, but each core is deep, pipelined, and optimized for making decisions fast. They handle branches, interrupts, memory lookups, and tool calls the way your prefrontal cortex handles driving in traffic.
When the dominant workload was training, GPUs won. When the dominant workload becomes agentic reasoning, the balance shifts.
The Orchestration Tax
Call it the orchestration tax: compute burned not on thinking, but on managing thinking, routing between tools, holding context, enforcing protocols, checking outputs, maintaining state across minutes of autonomous work.
In a 2024 training rack, you had roughly one CPU for every eight GPUs. That ratio reflected a world where the CPU’s job was mostly to keep the GPUs fed. In the newest agentic racks being deployed this year, the ratio is tightening fast — toward 1-to-2, sometimes 1-to-1. Not because GPUs got less important. Because the orchestration layer got massively heavier.
The Model Context Protocol — MCP, the emerging standard for how agents talk to tools and data, runs on CPUs. The agent-to-agent communication layer runs on CPUs. Vector database queries mostly run on CPUs. Memory management, scheduling, error handling, retries, all CPU work. The part of AI you actually interact with, the part that feels like intelligence rather than a chatbot, is a CPU workload wearing a GPU costume.
The Hardware Responds
Markets don’t reward the obvious for long. As the orchestration tax has grown, the chip designers have responded.
Intel, written off by almost everyone, buried a set of matrix math instructions called AMX deep inside its Xeon processors. AMX lets a CPU run inference on models up to about seventy billion parameters without touching a GPU. For a CIO deciding where to spend next year’s budget, this is not a footnote. It means the servers you already own can run the AI you want to deploy — no forty-thousand-dollar accelerator per socket required.
AMD took a different path. The Zen 6 architecture — Venice, if you like codenames — built neural tiles directly into the chip package, closer to the CPU cores than any external accelerator could sit. The performance-per-core gains are the kind of number that makes hyperscale procurement teams rewrite their roadmaps.
The sovereign silicon story is the most interesting of the three. Amazon’s Graviton. Microsoft’s Cobalt. Google’s Axion. All ARM-based, all custom, all designed to do one thing: reduce the Nvidia tax. Over half of new AWS workloads now run on Graviton. These chips exist because the hyperscalers got tired of shipping fifty cents of every AI dollar to Santa Clara. The ARM Scalable Matrix Extension means they can now handle serious inference. Quietly, ARM has become the most deployed AI architecture on earth.
Even Nvidia understands. Their Vera CPU, the successor to Grace, isn’t trying to replace the GPU. It’s trying to remove every bottleneck between the GPU and its memory. Nvidia’s own product strategy tells you where the constraint is, and it isn’t raw GPU compute anymore. The most valuable company in the world is building a CPU because even the GPU needs a conductor.
The Economics
All of this eventually lands on a spreadsheet.
A top-tier AI GPU in 2026 draws close to a thousand watts. A high-end server CPU draws two to three hundred. For workloads that don’t need to happen in the next millisecond, batch document analysis, overnight research agents, data pipeline work, running on a CPU cluster is forty to sixty percent cheaper than running on a GPU cluster. When you are pushing a billion tokens a day through a system, the difference between forty and sixty percent becomes the difference between a business that works and one that doesn’t.
Capital allocation inside data centers is following the math. Two years ago, eighty percent of server capex went to GPUs. This year, by most estimates, it’s closer to fifty-five. The shift is slow but structural. The inference economy, the part of AI that actually generates revenue, cannot afford the GPU premium on every query. It will route to whatever silicon does the job for less.
The ASIC Objection
The sharpest pushback on this thesis is that it doesn’t go far enough. If workloads get specialized enough that CPUs beat GPUs, why wouldn’t an even more specialized chip beat the CPU? Google’s TPUs, Amazon’s Trainium, Groq’s LPUs, Tesla’s Dojo, the ASIC logic has been right before, and it is moving up the stack. A reader who takes scaling laws seriously might argue that the CPU renaissance is a transitional window, and that custom silicon designed for agentic reasoning will close it within three years.
Parts of that argument are almost certainly right. Training is already drifting toward custom silicon, Trainium and TPU are eating share from Nvidia at the largest hyperscalers, and that trend will accelerate. Raw inference will go the same way for the workloads that matter most at scale. Groq and Cerebras are already cheaper per token than any GPU for stable model serving, and the economics only get worse for general-purpose chips from here. If the claim were that CPUs will win training or raw inference, it would be wrong.
But the claim is narrower than that, and it survives the objection. ASICs win where the math is stable, matrix multiplication, hashing, video decode, transformer inference. Stable pipelines reward being etched into silicon. The orchestration layer is the opposite of stable. Every week, agents talk to new APIs, parse new data formats, handle new error states, route through new tool chains. The heterogeneity is the point. You cannot tape out a chip for a workload that is redefining itself every sprint. For sixty years, specialized silicon has won inside pipelines and general-purpose silicon has won at the layer that decides which pipeline to run next. That pattern has survived every computing era so far.
The most likely outcome isn’t CPUs versus ASICs. It’s a three-layer stack: ASICs for training, ASICs for raw inference, and CPUs at the center holding the whole thing together. The CPU wins the orchestration layer, and that layer is the fastest-growing line item in AI compute.
The Trade
What follows is a framework for thinking about where value accrues in the CPU story. It is not a recommendation to buy or sell any security. Every reader’s situation is different; do your own work.
If you take this seriously, it changes what you own.
The obvious names, AMD for data center x86, ARM Holdings as the royalty collector on every custom hyperscaler chip, are the direct expressions of the thesis. Think of them as exposure to the orchestration layer, not to inference broadly. Some of that inference business will go to ASICs, and that is fine. The orchestration layer is the part that is structurally hard to specialize, and that is where the CPU franchises compound.
The more interesting money is usually in the second derivative. Advanced packaging, where chiplets get stitched together, lives at TSMC and a handful of specialists in CoWoS substrates. Liquid cooling, increasingly non-optional as core counts climb, is a quiet compounder. And Intel, if it can execute the foundry transition, becomes the only Western alternative to TSMC, a geopolitical call as much as a technical one.
The risks are real. A DRAM shortage would hurt the CPU story badly, because CPU inference is memory-bound in a way GPU training isn’t. CUDA’s software moat is genuinely wide, which means the transition will be slower than the hardware alone would suggest. And if a credible general-purpose ASIC for orchestration ever shows up, not for inference, for the messy work of tool-calling and state management, the whole thesis compresses.
But the direction is set. The training peak was real, and it built the foundation. What sits on top of that foundation — the agents, the reasoning systems, the autonomous workflows — is a different kind of workload, and it rewards a different kind of chip.
The Conductor
For three years we have been obsessed with the loudest instruments, the brass section of GPUs playing the same fanfare at massive scale. It was thrilling, and it deserved its moment. But an orchestra is not a brass section. An orchestra is a conductor holding a hundred different voices together, making decisions in real time about tempo and balance and when to bring the strings back in.
The CPU is the conductor. It was always the conductor. For a while the brass was so loud we forgot the conductor was there.
If your portfolio is still positioned for the training peak of 2024, you are listening for the brass. The next movement is already playing, and the conductor is back at the center of the stage.
Disclaimer
This post is for informational and educational purposes only. It does not constitute investment advice, a recommendation, or an offer to buy or sell any security. I, members of my family, or entities I am associated with may hold positions in the companies or sectors discussed, and those positions may change at any time without notice. Past performance is not indicative of future results. You should consult your own financial, tax, and legal advisors before making any investment decision.
Further Reading
A curated list of the primary sources sitting behind this argument. These are the pieces I returned to most while writing.
Patterson & Hennessy, “A New Golden Age for Computer Architecture” (Communications of the ACM, 2019) — the anchor paper for the domain-specific architecture thesis and the intellectual foundation of the ASIC objection. If you read one thing from this list, read this.
Jouppi et al., “In-Datacenter Performance Analysis of a Tensor Processing Unit” (Google, 2017) — the original TPU paper. Worth reading for how specialized silicon actually beats general-purpose hardware in practice, and what that implies for the orchestration layer.
Anthropic, Model Context Protocol specification (modelcontextprotocol.io) — the primary source on how agents connect to tools and data. Short, technical, worth skimming.
SemiAnalysis (semianalysis.com) — Dylan Patel’s ongoing coverage of hyperscaler silicon strategy, Graviton adoption, and the Nvidia tax debate. The best-informed source on what’s actually happening inside data center procurement.
Stratechery by Ben Thompson — the clearest ongoing synthesis of AI capex dynamics and their implications for platform economics and competitive structure.
Gavekal Research — for the broader capex rotation and energy infrastructure story that sits beneath this thesis, particularly on power and grid constraints.
Groq and Cerebras public benchmarks — the cleanest available data on inference-per-dollar for specialized silicon, and useful primary material for pressure-testing the ASIC objection.
Companies Mentioned
For reference only. Not investment advice.
Public: AMD, ARM, INTC, NVDA, TSM, AMZN (Graviton, Trainium), GOOGL (TPU, Axion), MSFT (Cobalt)
Private: Groq, Cerebras, Tenstorrent
Key Terms
AMX — Advanced Matrix Extensions. Matrix math units built into Intel Xeon CPUs for on-die inference.
ASIC — Application-Specific Integrated Circuit. Silicon designed for one workload rather than general computing.
CoWoS — Chip on Wafer on Substrate. TSMC’s advanced packaging technology for stitching chiplets together into a single package.
DSA — Domain-Specific Architecture. Silicon optimized for a particular class of workload. The Patterson/Hennessy framework.
HBM — High-Bandwidth Memory. Stacked memory sitting adjacent to GPUs for fast data access; expensive and supply-constrained.
MCP — Model Context Protocol. Emerging standard for how AI agents connect to tools and data sources.
SME2 — Scalable Matrix Extension 2. ARM’s matrix math capability, making ARM CPUs competitive for inference workloads.
TCO — Total Cost of Ownership. The full economic cost of running compute, including energy, cooling, and infrastructure, not just hardware.
Great article. Explains why Jensen tried so hard to buy ARM !