AI ·
History of AI
From the 1956 Dartmouth workshop to the transformer. The winters, the expert systems, and the turn in 2012.
The name is younger than the wish. In 1950 Alan Turing asked whether a machine could keep up a conversation well enough that a person could not reliably tell it from another person. The field got its letterhead in the summer of 1956, at a workshop at Dartmouth College. John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon had asked for a summer's work on "artificial intelligence." The summer ran long. The phrase stayed.
Two bets were already on the table.
The symbolic bet says intelligence is the handling of symbols. Write the rules, hand the machine the facts, and it reasons. Logic, search, and the expert systems of the 1980s all sit here. The machine is a clerk with a very large book of procedures.
The learning bet says intelligence is a pattern found in examples. Frank Rosenblatt's perceptron, in 1957, was one layer of adjustable weights. It could learn a line that separates two kinds of points. In 1969 Marvin Minsky and Seymour Papert published Perceptrons and showed what that single layer cannot separate, exclusive-or among the cases. The proof was about one layer. A large part of the field heard "neural nets are finished," and the money followed that hearing.
The first winter
By the early 1970s the promises had outrun the programs. The Lighthill report, written for the British Science Research Council in 1973, said so in plain language: the toy problems were yielding, and the real ones were not. In the United States, DARPA pulled funding back. Machine translation that had looked promising on a few sentences fell over on a paragraph. Search trees grew faster than the computers. A program that played a small game did not become a program that understood a page.
This is the pattern of a winter. The idea is not refuted. The invoice arrives before the machines, the data, or the tests can pay it. People who had budgeted for a decade of miracles cancel the budget. The work that continues does so quietly, in labs that still have a paycheck.
Rules, written by hand
The 1980s tried again with rules a person had written down. DEC's XCON, also called R1, configured VAX orders and saved the company real money. Firms bought Lisp machines to run the shells. Japan announced the Fifth Generation project, a national bet on logic programming. The product shape was a shell, a knowledge base, and an inference engine. A company would hire the expert, interview them for months, and type the result into rules.
It worked where the domain was narrow, the rules were stable, and someone stayed behind to edit them. It failed where the world was messy, where two experts disagreed, or where the expert had left. A rule base does not age into wisdom. It ages into exceptions. Around 1987 the market for the dedicated machines collapsed. That collapse is the second winter.
What kept moving in the cold is easy to skip, because it did not have a launch. In 1986 David Rumelhart, Geoffrey Hinton, and Ronald Williams wrote backpropagation up for a wide audience: a way to train a network with more than one layer by sending the error back through the weights. Pieces of the idea were older. The computers were small, and the labeled data was scarce, so the respectable toolbox of the 1990s was statistical. Hidden Markov models for speech. Support vector machines. Bayesian networks. Careful features, designed by a person, fed to a classifier that could be inspected.
In 1997 Deep Blue beat Garry Kasparov. That was search, an evaluation function, and hardware, aimed at one game with fixed rules. It was a milestone. It was a method for chess. Point it at a photograph and it has nothing to say.
The turn
The web brought the data. GPUs brought the arithmetic. In 2006 Hinton's group showed that a deep network could be trained if you were careful about how you started it. The public turn came in 2012. Alex Krizhevsky, Ilya Sutskever, and Hinton entered ImageNet with AlexNet, a deep convolutional network trained on GPUs. The error rate dropped far enough that the rest of the vision field changed tools within a couple of years. Speech followed. Then translation.
The other date I keep is 2017. Ashish Vaswani and colleagues at Google published "Attention Is All You Need." The transformer drops the recurrent loop. Every token can look at every other token in the window, and the looking happens in parallel, which is what the hardware wanted. Scale that architecture and you get the systems people mean by the word AI in ordinary speech: BERT in 2018, GPT-2 in 2019, GPT-3 in 2020. In November 2022 ChatGPT put an instructed, conversation-tuned model in front of people who do not read papers. The shock was the product. The papers were already five years old, and the bet was sixty.
What a programmer should keep
The name is a suitcase. Search, a logic program, a classifier, a language model, and a robot policy are different machines. Say which one you are buying. A client who says "add AI" has not said it yet.
Scale changed the economics. It did not retire evaluation. AlexNet mattered because ImageNet was a test other labs could rerun. A demo on a folder of your own photos is a demo. Ask what the number is, who measured it, and what happens on the examples the number left out.
The winters were about timing. The symbolic systems were real software that earned their keep in narrow rooms, and they stalled when the room got wider than the rule book. The learning systems were real mathematics in the 1960s, and they stalled until the data and the chips showed up. Both lineages are still in the building. A database constraint is a rule. A model is a pile of weights. A modern product is usually both, with a person on the decisions that cost money.
The next page is the stack as it is now: Modern AI.