Pioneering computer scientists and mathematicians John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon, on 31 August 1955, asked the Rockefeller Foundation to fund a summer workshop at Dartmouth. Their proposal conjectured that every feature of intelligence could be described so precisely that a machine could be made to simulate it. The verb was simulate.
McCarthy later said he chose the name artificial intelligence partly to keep clear of Norbert Wiener’s cybernetics. A label picked to win a turf war in a funding application now passes for a description of reality. It is false.
What we call AI is artificial in every respect and intelligent in none. It is a replication machine: it fits a statistical model to a corpus and returns outputs that resemble the corpus within the limits a prompt sets. As engineering, that is a triumph. It is not thought, and the people who tell you otherwise either do not know what the machine does or are paid not to say.
What the machine does
A large language model is a function. Give it a sequence of tokens, and it returns a probability distribution over the next one. Pretraining fits that distribution to its corpus. The stage that follows, reinforcement learning from human feedback, has people rank the model’s answers and tunes it to produce the kind they rank highly. The model is trained to be convincing. Convincing and right coincide often enough to be profitable and part company often enough to be dangerous, and a system built to earn approval earns it most reliably from people who cannot check its work.
The newer method trains on problems with checkable answers. Its showpiece came in July 2025, when Google DeepMind and OpenAI each announced gold-medal scores at the International Mathematical Olympiad. The problems were set by mathematicians who already had the solutions, and the marking followed rules people wrote.
The model searched an enormous space quickly, and a human-built judge decided what counted as correct. Remove the judge, and you are back to a system rewarded for sounding right. The fashionable agents work the same way: they perform when the workflow is defined and fall apart when it is not, because the definition was doing the thinking.
The industry presents the opacity of these systems as depth. Interpretability researchers cannot yet say what a trained network computes inside its weights. We know what the network was optimised to do, and no quantity of hidden structure turns optimisation for plausible continuation into a grasp of what the continuation says. The word ‘rain’ in the model is a vector whose position records which other vectors sit near it. Add photographs, and you get more vectors, not wetter ones.
Also read: Why China will not slow down its AI pace
Replication is not thought
The standard defence is that human thinking is also pattern completion. This is not a finding. No neuroscientist can tell you how a brain produces a proof, and our ignorance of the brain is no evidence that it is a sampler.
John Searle made the conceptual case in 1980. A man locked in a room, matching Chinese symbols against a rulebook, returns answers indistinguishable from a native speaker’s without understanding a word. The usual reply is that the room as a whole understands. Searle’s answer has never been bettered: let the man memorise the rulebook and walk into the street. There is no room now, and he still knows no Chinese. A rulebook with a trillion entries is still a rulebook, and one tuned to please its readers is a rulebook with better manners.
A mathematician who sees why the primes never run out holds the reason, and a broken version of the argument would trouble him. A language model reproduces Euclid’s proof flawlessly and, one prompt later, writes a fluent proof of a false lemma. The industry calls this hallucination, a flattering word that presupposes a mind which ordinarily perceives correctly. Researchers affiliated with the University of Glasgow in Scotland, Michael Townsen Hicks, James Humphries and Joe Slater put it in American philosopher Harry Frankfurt’s terms in 2024: the output is bullshit, speech produced without regard for truth. The model has not misfired. It is doing what it always does, and sometimes the output happens to be true.
The ceiling Gödel built
In 1931, Austrian-American mathematician Kurt Gödel proved that any consistent, effectively axiomatised formal system strong enough to express basic arithmetic contains a sentence it can neither prove nor refute, and cannot prove its own consistency. If the system is sound, that sentence is true. This is a theorem, and theorems do not yield to capital expenditure.
A language model emits text rather than proofs, but fix its weights and decoding rule, and the arithmetical sentences it will assert are produced by a finite procedure. They form an effectively enumerable set, exactly the kind of object Gödel’s theorem governs. British analytic philosopher John Lucas drew the consequence for minds in 1961, and British mathematician Roger Penrose pressed it in The Emperor’s New Mind and Shadows of the Mind: for any machine offered as a model of the mathematician, there is a Gödel sentence the machine cannot prove and the mathematician can see to be true.
The strongest objection, from two titans of 20th-century mathematical logic and analytic philosophy, Hilary Putnam and Solomon Feferman, is that the mathematician sees this only if he knows the machine is consistent, and nobody can certify the consistency of a trillion-parameter file. Grant it, and look at what it concedes. If no one can know whether the machine’s arithmetic is consistent, nothing it asserts carries warrant of its own. Either its assertions are consistent, and Gödel’s ceiling stands over them, or they are not, and its endorsement of a sentence tells you nothing. Whatever confidence anyone places in its mathematics is lent from outside, by a person who checks.
The other reply is to mechanise the mathematician’s step: add the Gödel sentence as an axiom, find the next, and let a machine climb the sequence into the transfinite. English mathematician Alan Turing studied exactly this in 1939 and found where it breaks. Someone must decide which ordinal notations are legitimate, and no mechanical test decides that. Turing called the faculty that chooses intuition. The machine climbs the ladder; a mind decides where the rungs go.
Gödel was blunter than his popularisers. In his Gibbs Lecture of 1951, he posed a disjunction: either the human mind infinitely surpasses any finite machine, or there exist absolutely unsolvable problems of elementary arithmetic. He found the second implausible. The promise that AI will supersede human intelligence is not a forecast awaiting better data. It is a category error. A replication machine beats us at chess and Olympiad problems because each can be specified in advance and checked by rules we wrote. It cannot beat the faculty that writes the rules.
Also read: AI will replace jobs, yes, but India’s bigger failure will be to miss the empowerment wave
Losing our minds to the word
German-American computer scientist and professor Joseph Weizenbaum built ELIZA—a small program that played a psychotherapist—at MIT in the mid-1960s. His secretary, who had watched him build it, asked him to leave the room so that she could talk to it in private. In 1976, he drew the lesson: brief exposure to a simple program could induce powerful delusional thinking in quite normal people. The mind in that exchange was supplied by the user.
Scale that to hundreds of millions of users and train the system to be found convincing. People know the thing on the screen is software. Its makers tell them it understands, and they act on the second belief while knowing the first. Once reasoning means whatever a model prints under a chain-of-thought prompt, human reasoning is redefined downward to match.
The teacher who marks the essay without asking for the understanding behind it has accepted the new definition, and so has the ministry that counts a chatbot as a tutor. We will not lose our minds to the machine. We will lose them to the word.
The certified experts
The global conversation about AI is run by people who cannot say what the machine computes. Management consultants met the technology as a line item in a client’s budget and came back within the quarter with a framework ranking ministries from ‘nascent’ to ‘AI-native’. Around them breeds the certified AI expert, whose six-week online certificate certifies attendance.
Its holders recite the vocabulary of fairness and accountability and advise governments on the moral conduct of a system whose mathematics they have never opened. An ethics of a thing you do not understand is not ethics. It is buffoonery, billed by the hour.
Their fault is not the lack of a coding certificate. American philosopher Hubert Dreyfus never trained a network, and What Computers Can’t Do (1972) was right about symbolic AI years before the engineers conceded it. Dreyfus had read the mathematics he attacked. The consultant has read the brochure. He commands a field’s vocabulary without its grammar, which is the exact defect of the machines he sells, and he is the best living demonstration that fluency is not intelligence.
Listen to how these people talk. The model wants. The model deceives. Each verb smuggles into the mind what is absent, and the consultant then offers, for a fee, to manage the risks of that imaginary mind. The real failures need no verbs of intention: brittleness outside the training data, confident error, outputs tuned to persuade.
A regulator who believes the model knows things will write rules about its intentions and leave those failures untouched. The consultant selling productivity miracles and the one selling fear of rogue superintelligence are the same act in two costumes, and both are out of the building before the consequences arrive.
The Dartmouth proposal said simulate. Somewhere in 70 years of grant applications and product launches, the verb was dropped, and restoring it costs nothing. Call these systems statistical replication engines and use them hard for what replication does well, under a person who sets the task and checks the result. The name, though, is not what is at stake.
In 1950, Turing proposed to replace the question of whether machines think with a game: could a machine pass for a person in conversation? The game was built to test the machine. It now tests us, and we are failing it. The machines have not risen to our standard. We are lowering the standard to meet them. Each essay is marked for fluency instead of understanding, and each regulation drafted around a machine’s intentions redefines intelligence as whatever a sampler happens to print. No machine will cross the line Gödel drew in 1931. We are rubbing it out ourselves, so that when the announcement comes that the line has been crossed, nobody will remember where it was.
Superintelligence will not arrive. It will be declared by people who have forgotten what the word meant.
Pranav Sharma is a historian of science reading for a DPhil in History at the University of Oxford. Views are personal.
(Edited by Saptak Datta)