Of all the AI buzzwords out there, “model” would seem to be free of hyperbole. Yet this innocuous-seeming word is being pressed to mean two contradictory things. A recent Harvard/MIT study of simulating planetary orbits illustrates the contrast between what scientists consider a “world model” and what AI enthusiasts think transformers are generating.
When trained to reproduce planetary orbits, AI jury-rigged an arbitrary set of predictions that happened to match the data. This is very different from boiling the laws of motion down to a few fundamental principles, which is required for a scientific model. More disappointing still is the fact that each transformer generated a different wacky force law depending on the data sample. Both state-of-the-art LLMs and a custom transformer trained specifically on orbital trajectories demonstrated the same incoherence.
A scientific model means something very different than a stochastic output simulator. It’s predictive, sure, but those predictions derive from a handful of generalizable principles that teach us something fundamental about the nature of the system, whether a hydrogen atom or a galaxy.
As an astrophysics major in college, I heard historian of science Owen Gingrich say something that blew my mind at the time. Copernicus’ original Sun-centered model of our solar system was hard to accept not just because it displaced humanity from the center of the cosmos, but also because it didn’t fit the facts as well as Earth-centered models.
Ptolemy and his descendants had gradually refined the geocentric model over 1400 years to match astronomical trajectories plotted by stargazers. Sure, it was hard to reconcile the movement of planets across the sky with orbits around the Earth, but not impossible. Ptolemy’s followers invented displaced orbits (“equants”) and mini-orbits (“epicycles”) for each planet; adding enough of these recursively made it possible to fit the data with increasing precision. The arbitrary complexity of these models, some with as many as 80 epicycles, is reminiscent of the complexity of neural networks.
According to Gingrich, Copernicus’s real revolution wasn’t finding a better fit to the data but finding a simple solution free of Ptolemaic curlecues—because it hinted at underlying laws that could be generalized to other scenarios. Copernicus’ model inspired Kepler to derive mathematical equations of planetary motion, which in turn became the inspiration for Newton to pen the laws that underpin all of classical physics. Copernicus’ aesthetic minimalism led not just to prediction, but to explanation and understanding.
Gingrich argued that a key requirement for such generalizability is mathematical brevity. The premise is that the wonders of nature, at least those we can comprehend, spin out from a few inherently simple rules—what my former classmate Brian Greene calls the “elegant universe.” All of classical electricity and magnetism, for example, can be boiled down to just two equations.
Today many heralds of artificial general intelligence are making the same mistake as Ptolemy and his followers. Because transformers can simulate a predictable world, we presume they are also generating models of it. But this confuses a set of universal axioms with a mechanism recursively refined to match data. Large language “models” do have many useful features, but compared to the mathematical models devised by human scientists, all transformers are overfitted to their data. The next Copernicus will not be an AI, though it may be a scientist working with one.
Top: The coherent force vectors proposed by the scientific model versus the arbitrary vectors proposed by transformers. Top right: James Ferguson, Representation of the apparent motion of the Sun, Mercury, and Venus from the Earth (1771). Bottom right: a three-layer neural network diagram (via Duif).
