In artificial intelligence, general methods that make use of more computation end up winning, and by a wide margin, over systems into which someone has built what they know about the problem. As Rich Sutton argues in The Bitter Lesson, this did not happen once. It has happened again and again over seventy years of research.
The reason is the price of computation. Each unit of computing costs less than the last, exponentially, yet researchers tend to work as if the available power were fixed. With fixed power, putting your knowledge of the problem into the system is almost the only way to improve, and it works: the system performs sooner, and it is satisfying to see your own understanding inside the machine. A few years later there is vastly more computation available, and that knowledge-laden system stalls while a simpler one, which only knows how to search and learn, overtakes it.
Chess is the clearest case. The machine that beat Kasparov in 1997 did not understand the game the way a grandmaster does: it explored enormous numbers of positions on specialised hardware. Researchers who had spent years programming chess knowledge received it with dismay, because that is not how a person plays. The same thing happened later with Go, with speech recognition and with computer vision, where hand-designed edge and shape detectors were dropped in favour of networks that learn from data.
The two families of methods that keep scaling with no limit in sight are search and learning. Neither depends on the domain: they work wherever there is enough computation.
In theory, leveraging human knowledge and leveraging computation need not exclude each other. In practice they do. Time spent on one is time not spent on the other, each approach creates personal attachments, and built-in knowledge complicates systems in ways that keep them from scaling. That is what makes the lesson bitter: what wins is exactly what looks least like how we think we think.
Underneath lies a broader idea. The actual contents of a mind, its space, its objects and their relations, are endlessly complex, and any simplification we impose falls short. What is worth building is not our discoveries, but the capacity to discover.