Human Unpredictability is Overrated
Follow into
Save into
Follow into
My good friend Michael Rose recently wrote an interesting piece on his Substack, Amplified Mental Arms Races Foster Free Will. There is much there worth thinking about, but towards the end he makes a statement that I object to:
The End of History: Surfing on Darwinian Chaos
The final corollary of MARA theory is a devastating one for the social sciences: no detailed predictable arcs in the rise and fall of civilizations. Detailed theories of history or politics inevitably crumble when faced with human creativity.
There are some minor regularities, such as those identified in Peter Turchin’s cliodynamics, as well as the market patterns that are studied by macroeconomists. But even those theories do not supply much detailed guidance with respect to major events like the onset of wars or the collapse of stock markets. Put another way, these theories are not entirely useless. They are, however, quite circumscribed. Sorry, Isaac Asimov!
I actually agree about the implausibility of predicting the dynamics of human societies very accurately very far in the future (as Hari Seldon did in Asimov’s Foundation). But the reason is not that humans are unpredictable, but because of mathematical chaos (sensitive dependence on initial conditions).
In fact, despite all the propaganda about free will, human beings are surprisingly predictable. Studies that followed smartphone use show that it’s possible to accurately predict not only physical locations, but also activities of people.
A recent study went even further (see A foundation model to predict and capture human cognition). These researchers built Centaur, “a computational model that can predict and simulate human behaviour in any experiment expressible in natural language.”
This is not to say that humans are completely predictable. People’s behavior has both a highly structured component, and an unpredictable, stochastic component. Different behavioral domains are characterized by a highly variable mixture of order and randomness. Thus, routine behavior (that you will be in office on Monday at 9:00 am) is highly predictable. But major life decisions are much less so. For example, it’s difficult to predict who will commit suicide tomorrow. Overall, though, in most domains determinism is more prevalent than stochasticity.
So, what’s wrong with Michael’s logic — why are humans mostly predictable, despite his argument? I agree with him that unpredictability is an advantage in adversarial, competitive interactions. But the opposite is true for cooperative actions, or in situations that require coordination.
(A technical note: in game theory cooperation is when agents need to sacrifice personal advantage to concert collective action; whereas coordination involves no costs to participants. An example of the latter is agreeing on which side of the road to drive — following this rule is costless.)
I don’t want to cooperate with someone who behaves erratically. And most of our day-to-day dealings are in the cooperation/coordination domain — at least, 90%, perhaps even 99%.
As an employer, I have, on numerous occasions, had to let go unpredictable employees — those who don’t show up on time, don’t perform expected tasks, or quarrel with colleagues.
Of course, I don’t operate an industrial assembly line — I also expect new, creative solutions from people in my group, but that’s maybe 10% of their work, while the rest is routine.
As an illustration how the nature of interaction, competitive or cooperation/coordination, shapes your behavior, consider judo and dancing. I practiced judo as a young man for more than ten years, and the first thing you learn is not to be predictable. As soon as you fall into a pattern, you get thrown. Dancing, by contrast, is a highly patterned activity — you need to be completely predictable to your partner (or partners, if it’s something like the square dance).
So much for unpredictability of individual humans. When we shift the focus to large human groups, in many cases our ability to predict the behavior of such collectives becomes much better than that of individuals. Thus, an individual suicide is quite unpredictable, but at the level of the United States, the number of suicides by Americans in 2026 is predictable with a small margin of error. I’ve written on this issue before: see Psychohistory and Cliodynamics. Furthermore, prediction is overrated: see Scientific Prediction ≠ Prophecy.
Therefore I completely disagree with Michael when he says,
There can be no long-sustained pattern to human history because we are constantly innovating our way out of our own structures. We are surfing on Darwinian chaos.
Michael’s view is similar to the arguments against scientific history, formulated by the philosopher Karl Popper. Popper’s main point was that because the future course of human history is critically affected by the development of knowledge, and because future scientific and technological discoveries cannot be predicted, a predictive science of human history is in principle impossible.
It is ironic to find Michael Rose agreeing with Karl Popper, because Popper also criticized evolutionary biology, which, in his view was not a real science, but at best “a metaphysical research program.” I wonder whether Michael agrees with Popper’s negative assessment of his own discipline!
Even Popper’s claim that future scientific discoveries are completely unpredictable is overstated, if not simply wrong. Any discovery that becomes part of science is a product of an individual scientist’s brilliance and the social milieu that nurtures it and is ready to accept it. This is why Gregor Mendel’s discovery of the basic principles of heredity was ignored in 1865, and then rediscovered simultaneously by three geneticists in 1900, who published articles within two months of each other. Simultaneous discoveries of the evolution by natural selection, calculus, and many others, further prove the point.
As for technological progress, just look at Moore’s Law, which has been operating for decades.
History is not “just one damn thing after another,” as Arnold Toynbee once said. Rather, human societies are just like any other dynamical systems; their behavior is a mixture of stochasticity and determinism. I’ve been analyzing quantitative data on the dynamics of states and empires now for three decades, and I constantly encounter new examples of remarkably regular patterns. Perhaps one day I’ll gather them in a list, but you can actually see many such instances in my posts on this Substack and the Blog Archive, most recently in this post. At the same time, history also has a lot of “damn things.” But the domain of unpredictability is shrinking as our science of societies matures (although, of course, it’ll never shrink to zero).