Anthropic has brought out its prospectus. Revenue is climbing, token consumption is soaring, and people are talking about an AI bubble again. The financial questions are familiar: Is demand big enough? Can revenue cover spending? When will all those data centers pay for themselves? The technical questions are familiar too: Are models still improving? Have their abilities hit a ceiling? How far away is AGI?
I come to this as a humanities graduate and a designer. There is another way I can imagine the bubble bursting.
I worry that we are mistaking AI’s astonishing but uneven abilities for a general intelligence that will soon be able to serve everyone.
The infrastructure boom rests on a belief that AGI will eventually meet almost every kind of demand. If we discover that today’s AI is still nowhere near that goal, much of this spending could become a sunk cost. The fiber laid during the dot-com bubble could at least stay useful for another decade or more. What about AI chips? They are, in effect, consumables that need replacing quickly.
The progress in coding, mathematics, and science is remarkable. But most people’s work and lives involve more than problems that yield to rigorous reasoning. Can AI understand what someone actually needs? Can it handle an everyday problem that is hard to articulate and has no correct answer? Can it produce a genuinely inventive solution when you cannot see a way through?
On those tasks, the AI I use is still a long way from the AI in the demos.
Brilliant, and bafflingly clueless
Working with AI often feels like working with someone whose IQ is over 500, but whose maturity has not made it past elementary school.
It can explain an extraordinarily complicated problem and solve it efficiently. It just does not seem to know why you wanted it solved.
I maintain two personal projects. One helps me organize work and life, set goals, and track them. It keeps adding tasks, but never finds a way out of the impasse that made me seek help. The other helps my daughter practice reading and writing in English. I want it to explain things clearly. Instead, it clings to its explanation, repeating terminology she does not understand. It never quite becomes a teacher.
It does every task. Every step is careful and logical. The most important question goes unanswered: Is any of this helping? Are we actually solving the problem?
The more I use it, the clearer the difficulty becomes. Execution is rarely the hardest part. Knowing when to try something else, do less, or stop and reconsider is.
AI is very good at following a request through to completion. It is much less good at understanding why a person would make that request, or what they hoped to achieve.
In creative work, it can give me plenty of plausible, fully developed proposals, each with a thorough explanation. Yet after years of using it, I can honestly say it has never given me an idea that made me sit up. It will reason further and further down the wrong path. The impasse that needs breaking remains untouched.
As I understand it, the AGI we imagine should be able to do the vast majority of the world’s jobs. I have been paying for AI subscriptions since 2023. What I have seen so far is a technology especially good at coding and scientific work that demands sustained reasoning. Most work in the world does not depend primarily on that kind of reasoning.
More parameters may not fix this
This is not a complaint based on a handful of casual questions. I have used models from Anthropic, OpenAI, and Grok extensively. I have tried detailed context constraints, agent harnesses, and checks across multiple models. I have built projects on Grok Bot’s cloud computer, with Claude and Codex taking turns doing the work and checking each other’s results. Their weaknesses are strikingly similar.
The improvement in coding is unmistakable. Tools that once helped write a few snippets can now complete complicated projects. I have seen no comparable advance in messy human affairs or creative work. The prose is smoother. Parameter counts have gone from 175 billion to a rumored 10 trillion. Yet the models still get confused in the same places. On these tasks, their abilities seem barely to have moved.
That makes me wonder whether the basic technical approach behind today’s models is particularly suited to reasoning, and much less suited to work that depends on understanding people.
Does rapid progress in one ability really mean that other abilities will improve at the same rate?
My experience is of powerful individual capabilities that still do not reliably add up to a job done. Understanding the goal, finding an inventive solution, checking the result, and correcting mistakes: much of that work still falls to me.
Sometimes it feels as though AI makes three mistakes for every two tasks it completes. It does some work for me, but creates plenty of problems for me to sort out. The net time saved may be smaller than people suggest.
AI has already created enormous value in some kinds of work. I do not dispute that. But learning to code does not automatically supply all the missing abilities between here and doing most of the world’s jobs.
That is where I think the bubble may be: we have taken extraordinary performance on some problems as an early sign that AI will soon handle every problem. The bubble could burst when people think carefully about that leap.
Scientists need flashes of insight too
At the highest level, science needs flashes of insight too. So far, I have not seen AI show that ability at all.
Give a competent engineer a clear objective, and they can use existing knowledge to reach it step by step. An exceptional engineer sometimes has to ask a different question first: Is the route everyone takes still worth following? Some solutions also need elegance. They need taste.
The same is true of mathematicians and physicists. Extending an established line of thought and suddenly seeing a different way to understand the problem are different acts of thinking. The closer you get to the frontier, the more you may need an insight that cannot be scheduled in advance.
I encounter this in design all the time. I work through every constraint and still cannot solve the problem. Then I see it differently, and one simple move resolves several difficulties that had seemed hopelessly tangled. Afterward, I can explain why the solution is good. Explaining the chain of reasoning that produced the insight is much harder.
Sometimes I think of inspiration as a crossed wire in the brain. Most crossed wires produce errors. Occasionally, one connects things that had no apparent connection, and an unexpected answer happens to work. AI’s rigorous reasoning can seem almost too correct to allow that sort of creativity.
That occasional leap is precisely what I want from AI in creative work. Instead, it usually keeps circling within the same idea, more diligently and more exhaustively, without ever stepping outside it on its own.
This is why I do not believe AGI is coming soon. My suspicion is that crossing this gap will require a deep revision of the principles behind today’s models, possibly a rebuild.
If AI could reliably meet most people’s needs most of the time, I would consider today’s enormous investment justified. Almost everyone’s work and life would be its market. That is certainly big enough.
But most people are not programmers. Much of what they deal with each day is complicated, ambiguous, bound up with relationships, and less than rational. In my experience, AI is still nowhere near something I can trust to handle those matters.
My doubt is about the timing of the bet. We have already invested for a future of general intelligence. That future may be so far away that every investor loses patience before the returns arrive.
What is human inspiration actually worth?
AI can reason but cannot escape its own logic. It can execute but often misses the purpose. It knows a great deal but mishandles subtleties that people understand. When real invention and insight are required, humans still have something AI cannot learn.
At this point, a humanities graduate can breathe a little easier. The abilities I know and value, even if I struggle to explain them, still occupy the higher ground.
Follow that argument far enough, and I seem to have found AI’s fatal weakness.
Have I?
Let us try one more of those supposedly human flashes of insight, this time on the argument itself.
A thought occurs to me. When someone in Qing-dynasty China first saw machinery from the West, did they feel something similar? Those machines were impressive, certainly. But the highest culture, cultivation, and understanding still belonged to us.
What makes us so sure we are not doing the same thing now?
I say AI does not understand relationships, cannot have sudden insights, lacks inspiration. But do those abilities seem so important partly because we spend our lives dealing with problems peculiar to humans?
As a designer, much of my creativity goes toward making a product feel easier and more pleasant for another human to use. I turn needs that a person cannot articulate, or that contradict one another, into a solution that makes sense to them. I untangle the complications: habits, feelings, culture, and all the less-than-rational things people want.
What happens if we take humans out of the picture?
No relationships that need interpreting. No conflicting, tangled desires. None of the complicated problems that exist only within human society. Why would a world made entirely of AI keep concerning itself with those things?
I tend to believe the universe has a rigorous logic of its own. It does not have to accommodate human feelings. A silicon-based civilization would not need to think and live as we do. Could it advance more efficiently by developing the abilities it already does best? Would the flashes of inspiration we prize still be indispensable to it?
When machines learned to calculate, we said only humans understood language. When they learned to write, we said only humans had aesthetic judgment. As they take another step, we point to creativity, inspiration, and insight. Are the things they cannot yet do also becoming the last places where we can bear to feel superior?
We think we are judging whether AI is intelligent enough. But is the yardstick we use a higher standard, or simply the shape of our own stubborn, muddled minds, mistaken for a universal measure?
I began by asking how far AI is from the intelligence I need, and whether the bubble will burst.
Now I have another question. Why should the AGI we want be the kind of intelligence AGI ought to become?