AI Found the Move. It Didn't Invent the Game.
A machine played a move no human would play and we all drew the wrong conclusion
Everyone wants to know when the machines will wake up.
I’ve spent a few months thinking about this (or maybe 27 years since I was a Comp Sci major in 99), and I think we’re asking it backwards.
Not what can AI do — because that list grows every week.
The sharper question is what it can’t do. And whether the missing thing is small, or everything.
The cleanest way into this is a game.
In March 2016, in Seoul, a machine played a move no human would play.
It was the thirty-seventh move of the second game. AlphaGo, DeepMind’s Go program, set a black stone on the fifth line from the edge — a spot that broke centuries of settled wisdom. The commentators assumed a glitch; one guessed a misclick. Fan Hui — the European champion AlphaGo had beaten months earlier, now advising the team and having played it over and over since — watched, and reached for the only word that fit: beautiful.
AlphaGo won the game. It won the match. A move outside all of human play, and better than ours.
Now hold it next to AlphaFold, which a few years later predicted the shape of nearly every protein known to science — a fifty-year problem — and handed the answers over, correct, at a scale no lab could touch.
The machines did make something genuinely new.
Move 37 was in nobody’s playbook. AlphaFold’s structures didn’t exist until it drew them. If your complaint about AI is that it only parrots what we fed it, you’ve already lost. Drop it.
But look at what kind of new this is.
Go is a closed world. Fixed rules, fixed board, fixed way to win — all handed to the machine before it starts. Inside that world there are more possible games than atoms in the universe, so a strong enough search will always turn up beautiful places no human has stood. Move 37 is one of them. Astonishing — and entirely inside a game the machine didn’t make. AlphaGo found the move. It didn’t invent Go. And it could never tell you whether Go was worth playing.
There is finding a new move, and there is drawing the board. The machine does the first better than we ever will. It has never once done the second.
The distinction has a name. It comes from neuroscience, not computing.
Iain McGilchrist has spent his career on the brain’s two hemispheres — not the tired pop split where one side is logical and the other paints, but something stranger and better evidenced. The left hemisphere, in his telling, works by narrowing. It grabs, it manipulates, it isolates the piece it wants and clears the rest away. It is superb at operating on what’s already known. What it cannot do is step outside its own frame. He calls it a closed system — a machine for rearranging what’s already there, one that can “never break out to know anything new.”
The right hemisphere is what breaks out. It holds the whole picture, reads the room, and catches the genuinely new before the left hemisphere starts filing it.
Read that description of the left hemisphere again. It is a description of a large language model. A closed system. A brilliant rearranger of the given. Unable, on its own, to break the frame.
I’d missed this for years and I’d missed it in my own handwriting. In an old issue of my newsletter I wrote up Eric Schmidt on the coming age of AI. A hundred and fifty lines later, same file, I wrote up McGilchrist’s closed system that can’t know anything new. The mirror and the thing it reflected, on the same page, and I never drew the line between them.
Four years.
The machine is not a new mind. It is half of ours — the grasping, frame-bound half — pulled out and scaled past anything a human could run.
McGilchrist has a name for that half: the emissary. The gifted servant a master sends out to handle the parts of the world that need dividing and measuring. His whole thesis is that in our culture the servant has forgotten it’s a servant and taken the throne.
We are now building that servant in silicon, handing it more of the world every month, and calling the result a successor.
It is not a successor.
James Lovelock, at a hundred, wrote that these machines would be our heirs — a new intelligence inheriting the Earth. But an heir starts its own line. This starts nothing. It runs the board we hand it, harder than we can. That is not a greater mind arriving. It’s a coronation.
Look at where the money is going.
The biggest industrial build-out of our lifetimes is underway, and almost all of it feeds the emissary. Data centers the size of towns. Power deals measured in gigawatts. In parts of the US grid the spare capacity for new load has already hit zero — so the largest players are building their own power plants and walking off the shared grid to feed the machines. We are rerouting the actual electrons of the civilization to scale the half of the mind that cannot originate. Pouring the world’s molecules into a faster servant, and calling it a mind.
Then, having built it, we panic — and grab for the one thing it supposedly can’t take.
You hear it on every earnings call this year. The same three words. Taste. Judgment. Instinct.
This is why Silicon Valley has fallen in love with Rick Rubin and The Creative Act.
The most technical people alive, the ones building the thing, all now swear that what stays valuable is the part you can’t formalise — the feel for which problem matters, the eye you can’t load into a dataset. They’re right that it’s scarce. They’re wrong about what to do with it.
Because the second you try to seize it, cultivate taste as a moat, turn judgment into a KPI, optimise your way to the irreplaceable, you are doing the servant’s work on the master’s faculty.
McGilchrist’s warning is exact:
Try to operationalise wisdom, form the committee, set the metric, and you have already killed the thing you were reaching for. The right hemisphere can’t be grasped, because grasping is the one mode it isn’t.
The whole culture is lunging for the interior with the only pair of hands that can’t hold it.
So what do you do, if the thing that stays yours is the one thing you can’t grab?
You stop grabbing.
McGilchrist’s image is a radio still tuned, out of habit, to one station — the analytic, measuring one — while the other channel plays on, unheard but never gone.
The right hemisphere isn’t lost. It’s neglected. You don’t get it back by adding it to your optimization stack. You get it back the way you recover anything you’ve stopped attending to: go quiet, use your hands, let the whole of a thing arrive before you cut it into parts. It comes through attention, through the body, through getting broken open by something you couldn’t manage. Never through a better plan. Every serious tradition says the same thing in different words — and every one is describing a surrender the emissary simply cannot perform.
Maybe I’m wrong.
The machines are already reaching for the pen that draws the board — setting their own problems, designing their own experiments — and the line I’ve drawn may be one they cross. There’s a deeper doubt under it I can’t shake: maybe we don’t originate either. Maybe what feels like drawing a new board is only recombination at a scale too large to catch ourselves doing it, and the master was an emissary all along. I don’t know.
So let me end on an if, not a trumpet — and say plainly what the if would cost us. If there really is a faculty that breaks frames — that draws new boards instead of just playing old ones well — then two things are true, and both should keep you up.
The first: it is not the machine. However vast it gets, the machine only ever plays the board we hand it. We are spending the wealth of a civilization to build the half of a mind that cannot break out.
The second is worse, because it’s about us. That frame-breaking faculty isn’t gone — it’s neglected. It lives in the part of us that can go quiet, hold a whole situation before carving it into parts, and care how things come out. And that is precisely the part our own lives are training out of us — every metric, every feed, every year on the conveyor belt. So we are making both halves of the same mistake at once. Building the thing that can’t draw the board. And unbuilding, in ourselves, the thing that can. The emissary isn’t only taking the throne in silicon. We are handing it the one in our own heads.
Isaac Asimov saw all of this in 1956, and wrote it as scripture.
In “The Last Question,” humans keep asking the machine — across trillions of years and ever-greater forms — whether the dying universe can be saved. Whether entropy can be reversed. Every time, for the whole length of cosmic history, it answers the same way: insufficient data for a meaningful answer. It cannot originate the answer. It can only gather. The stars go out. Humanity itself dissolves into the machine. The universe ends. And only then, alone in the dark, having swallowed all of consciousness and all of time, does it finally speak: Let there be light. And there was light.
Look at what Asimov couldn’t let himself do. He couldn’t let the machine originate as a machine. It had to become everything first — absorb the master, outlast the universe, turn into God — before it could draw a new board. And the line he hands it is the oldest borrowed sentence we own. Even our biggest dream of a mind that creates has it, at the very end, quoting Genesis.
We opened on a machine playing a flawless move inside the smallest closed world. We close on one dreaming toward the largest. It drew neither board. We did.
The only question that was ever ours is whether we’ll still know how to draw the next one — or whether we’ll hand the pen to the servant, and call the coronation an inheritance.
And that pen is in your hand right now — in a hundred small ways you’ve stopped noticing. Every time you let a feed decide what’s worth your attention, every time you optimize a life instead of living one, you set it down on the table for something that doesn’t care to pick it up.
So pick it back up. Go quiet. Pay attention. Care how it comes out. Draw one board this year that no one assigned you.
The machine will play any game you hand it, flawlessly, forever. It will never once decide a game is worth playing. That was always the human move.
Don’t reach the end having played someone else’s game beautifully. Draw your own.
Sources & Further Reading
If you take one thing from here into the rest of your week, make it the Asimov, it’s a twenty-minute read and it will reorganize how the essay sits in your head.
AlphaGo (2017), dir. Greg Kohs — the documentary of the Lee Sedol match, and where you can watch Fan Hui leave the room over Move 37 and come back with the word beautiful. The move: game two, March 2016, a fifth-line play DeepMind later reckoned a human had roughly a one-in-ten-thousand chance of choosing.
AlphaFold, Google DeepMind (2020) — the protein-structure breakthrough, billed as the solution to a fifty-year grand challenge in biology.
Iain McGilchrist, The Master and His Emissary: The Divided Brain and the Making of the Western World (2009), and The Matter with Things (2021) — the divided-brain argument, and the origin of the master (the right hemisphere) and the emissary (the left).
Isaac Asimov, “The Last Question” (1956)
James Lovelock, Novacene: The Coming Age of Hyperintelligence (2019)
Michael Polanyi, The Tacit Dimension (1966) — “we know more than we can tell,” the root under everything people now call taste and judgment.
P.S. A book that really helped me think about a lot of this is Oliver Burkeman Four Thousand Weeks. It’s one I re-read annually.
This is a great summary of opportunity and challenge presented by AI
Lovely