Statisitcal mechanics as an example of cultural evolution

John O. Campbell

In May I made a post titled 'Statistical Mechanics' which is an excerpt from my book Darwin Does Physics. It has proven to be the most popular post on this blog so I decided to post a further excerpt from the book that may provide context.

A Unified theory of Behavioural Sciences

The behavioural sciences, which attempts to explain and describe the processes of human and animal behaviour from many different perspectives, contain a plethora of evolutionary theories. These various perspectives have generated a tower of babel of overlapping jargon and concepts. Many employ a Darwinian/Bayesian approach.

Some behavioural scientists, such as the economist Herbert Gintis, have proposed a single Darwinian/Bayesian theory to encompass the many different schools of the behavioural sciences (1). Gintis calls his theory the Beliefs, Preferences, and Constraints (BPC) model and bases it on two basic processes: Darwinian evolution and Bayesian game theory.

Gintis acknowledges the reliance which both neural and cultural mechanism have on biological mechanisms resulting in the view that cultural is a nested hierarchy of evolutionary processes. He draws upon the gene-culture co-evolution model of cultural evolution which explains the interplay of cultural and biological mechanisms within this arena.

Figure 11: Economist Herbert Gintis is one of a number of Social Scientist to propose a unified theory of cultural evolution.

Darwinian component

The BPC model views social learning between generations or the process of socialization as the dominant Darwinian process of cultural evolution. It is a process employed by all cultures and has the hallmarks of a Darwinian process: there is copying of behaviours and beliefs between generations, there is variation amongst these copied beliefs and behaviours, and there is selection. 

Cultural transmission generally takes the form of conformism– that is, individuals accept the dominant cultural forms, ostensibly because it is fitness-enhancing to do so. Although adopting the beliefs, techniques, and cultural practices of successful individuals is a major mechanism of cultural transmission, there is constant cultural mutation, and individuals may adopt new cultural forms when those forms appear better to serve their interests.

Bayesian Component

The other pillar of Gintis’ theory is Bayesian game theory which Gintis views as the selection mechanism for his evolutionary theory.

The analysis of living systems includes one concept that does not occur in the nonliving world and that is not analytically represented in the natural sciences. This is the notion of a strategic interaction, in which the behavior of individuals is derived by assuming that each is choosing a fitness-relevant response to the actions of other individuals. The study of systems in which individuals choose fitness-relevant responses and in which such responses evolve dynamically, is called evolutionary game theory.

In this view individuals and cultures jockey for Darwinian success on the basis of their preferences and beliefs. While our beliefs may be highly uncertain or inaccurate, evolutionary game theory allows us to do the best we can with the beliefs we have and maybe even to occasionally update our beliefs. The Bayesian nature of Gintis’ Belief, Preferences and Constraints theory is reflected in his treatment of Beliefs.

It follows that beliefs are the underdeveloped member of the BPC trilogy. Except for Bayes’ rule, there is no compelling analytical theory of how a rational agent acquires and updates beliefs, although there are many partial theories

Thus in the terms of this theory cultural evolution is a Darwinian process where selection is made amongst human variations in beliefs and preferences on the basis of the consequences these have. It is a fundamentally conservative process in that we are loathe to change beliefs and preferences inherited from the previous generation.

Because cultural beliefs tend to be inherited and their relationship to the evidence is not examined too carefully, cultural evolution may seem excruciatingly slow. Much of this inertia may be due to the faith we place in those social beliefs we learned as children. Faith is an inherently illogical component of cultural evolution for faith implies a commitment to belief regardless of the evidence at hand. While this kind of conservative strategy may make sense in many situations it is illogical and inefficient if we have sufficient evidence to accurately assess the beliefs we have inherited.

In many settings a conservative approach makes good sense. If our ancestors have passed down a set of behaviours with proven survival ability it would be unwise to make random changes. 

Although it is generally inefficient and conservative, culture acts as an inferential system where models based on beliefs are updated on the basis of their experience in the world.  There are time lags, often on the order of many generations, between new evidence and the changing of beliefs. Often the updating of beliefs occurs only on the basis of survival, when one culture replaces another.

A simple example in which the evolution of a cultural processes is inefficient is supplied by a study of traditional Indonesian seaweed farmers. The study concluded that over years of experience the farmers had found the near optimal spacing of seaweed pods but they had failed to find the optimal size for the pods (2). In Bayesian terms, the evolution of their farming technique was sub-optimal because it failed to gather evidence on an exhaustive hypothesis space.

It is highly plausible that a theory resembling Gintis’ will eventually succeed in providing a comprehensive explanation of cultural evolution. Possibly human cultures will become more adept at updating themselves in an efficient Bayesian manner. The evolution of science provides an example of a cultural process whose efficiency as a inferential system has evolved over time.  

Perhaps science is the cultural process with most freedom from the inertia of faith. While scientists are often loathe to abandon their beliefs, the scientific method dictates that gut-feelings, intuitions or authoritative ancient texts carry little weight in the face of clear evidence; beliefs must be changed to conform with the evidence.

While the concept that hypotheses must be selected on the basis of evidence has been fundamental to science since Bacon the implication was unclear, at first, that scientific hypothesis are uncertain and are best described with probabilities. Understanding the role of probability and Bayesian inference within science is a central theme of its history. Pascal and Fermat introduced some probabilistic ideas to mathematics during Newton’s lifetime but only for the purpose of analyzing gambling games. In 1713, eight years after Jacob Bernoulli’s death, his great work on probability was published which extended the mathematics of probability and suggested some scientific applications. Probability only became part of the scientific tool box in 1774 when Laplace independently discovered Bayes theorem and used it to solve some outstanding scientific problems such as the ‘great Jupiter–Saturn inequality’ (3).

However even Laplace used Bayesian inference merely to resolve ambiguities in data rather than as an essential element of the scientific method. It was not until the mid-nineteenth century that James Maxwell introduced probabilities as a fundamental concept in science when he proposed an equation which described the velocity of gas molecules at a given temperature in terms of a probability distribution. His work was extended by Boltzmann and Gibbs into the field of statistical mechanics which was able to derive, using what turned out to be Bayesian inference, the many findings of thermodynamics from the kinetic or atomic theory of materials.

Beginning in the 1870s Francais Galton introduced probabilistic and statistical methods for the purpose of analyzing biological variations. His methods were later developed by Karl Pearson and R.A. Fisher, largely in a biological context, within the frequentest school of probability.

Unfortunately Galton, who was the cousin of Charles Darwin, was-over enthusiastic in his understanding of the power which genetics has for molding humans. He tended to believe that the human race might be improved if its fittest members were encouraged to have a greater number of children. He coined the word ‘Eugenics’ which developed into a barbarous cultural influence responsible for, amongst other atrocities, the castration of low IQ people who fell into the hands of the authorities in many countries including Belgium, Brazil, Canada, Japan and Sweden. The study of Eugenics may also have supported notions of racial superiority which were central to some of the worst forms of 20^th century ideology including Nazism.

Although Eugenics developed into a monstrous cultural practice, Galton’s understanding of Eugenics was much more restrained and humane (4):

Galton invented the term eugenics in 1883 and set down many of his observations and conclusions in a book, Inquiries into Human Faculty and Its Development. He believed that a scheme of 'marks' for family merit should be defined, and early marriage between families of high rank be encouraged by provision of monetary incentives. He pointed out some of the tendencies in British society, such as the late marriages of eminent people, and the paucity of their children, which he thought were dysgenic. He advocated encouraging eugenic marriages by supplying able couples with incentives to have children.

However, the ugly racial scar to which the development of Eugenics contributed should not negate the central contributions which Galton made to the development of biology. By introducing probability and statistics to the study of biological systems he provided the central tools which would be used in constructing biological models. Thus he should be considered one of the founders of modern biological science, in particular the field of bio-metrics.

Figure 12: Francais Galton introduce probability and statistics to the study of biology.

Later in the first half of the twentieth century probability was found to be a core component of the quantum description of matter.

In this manner physics and biology underwent revolutions with the introduction of probabilities and models were introduced. Although it was not immediately clear, each of these innovations can be understood as examples of inferential systems. It is interesting that at the early stages of their discovery each of these inferential systems were widely thought to be only abstractions or methods of calculation; they were not thought to have an actual existence in reality.

This is still the consensus view of quantum theory but it may be surprising that atomic theory was also in a similar state of limbo for nearly a hundred years. During the nineteenth century the physics establishment rejected the idea that atoms could actually exist and many did not believe that any direct experimental evidence would ever be found for anything so small as atoms. This was inconvenient for Ludwig Boltzmann who was attempting to derive thermodynamic results by modeling materials as large collections of atoms (5):

He had a long-running dispute with the editor of the preeminent German physics journal of his day, who refused to let Boltzmann refer to atoms and molecules as anything other than convenient theoretical constructs. Only a couple of years after Boltzmann's death, Perrin's studies of colloidal suspensions (1908–1909), based on Einstein's theoretical studies of 1905, confirmed the values of Avogadro's number and Boltzmann's constant, and convinced the world that the tiny particles really exist.

At about the same time that Boltzmann faced disbelief and scorn over the existence of atoms Gregor Mendel published a little noticed paper which described his experimental results in pollinating and growing 5,000 pea plants. His noted a law in his results; when a plant having a dominant characteristic, say a purple flower, is crossed with a plant having the recessive characteristic, say a white flower, their offspring will all produce purple flowers, the dominant characteristic. If these second generation purple flowered plant are then crossed their offspring will produce both purple and white flowers in a ratio of 3:1.

Mendel referred to an inherited ‘factor’ which distinguished between purple and white flowers in each generation. In modern terminology his term ‘factor’ has been replaced with ‘gene’. At the time his findings were thought both limited and unimportant and were ignored for several decades. However the problem of inheritance was one of the main gaps in the developing Darwinian theory of evolution. Many biologist assumed that inherited traits would tend to blend, that crossing purple and white flowered pea plants should result in light purple offspring. Darwin himself favoured a theory of ‘pangenesis’ which failed to hold up to experimental examination.

In their search for clues to the nature of biological inheritance some researchers re-discovered Mendel’s paper and were able to reproduce his experiments. Mendel’s long forgotten work rapidly became a center-piece of biological theory.

A heated debate then occurred as to the compatibility of Mendel’s and Darwin’s theories. Whether a leading biologist did or did not believe that Mendel’s theory was compatible with Darwin’s they shared a consensus that Mendel’s factors or genes had to be understood as a calculational device; they did not have an actual existence (6).

Of course both atoms and genes are now understood to actually exist but we might wonder about the heated initial resistance to these concepts. I suggest that one reason for the disbelief is that Boltzmann and Mendel assigned probabilities to atoms and genes. Probabilities meant uncertainty which for science, dominated as it was by the certain clockwork mechanisms of the Newtonian paradigm, was almost blasphemous. Much better that theories involving probabilities should be considered mere approximations or aids to calculations until the real deterministic theory is found.

We have also seen that where there are probabilities there is also information, entropy and an internal model; the existence of a probability implies an entire inferential system.  This was a lot for science to bite off especially at the beginning when only the tip of the ice berg was yet discovered. Biology has successfully made the transition. The Neo-Darwinism model of biology is the scientific consensus and clearly describes an inferential system independent of human calculation.

Boltzmann

Perhaps Boltzmann’s story and the development of statistical mechanics based on atomic theory is even more interesting. Surprisingly Boltzmann was a big fan of Darwin (7). 

If you ask me about my innermost conviction whether our century will be called the century of iron or the century of steam or electricity, I answer without hesitation: It will be called the century of the mechanical view of Nature, the century of Darwin.

Boltzmann understood that Darwin had constructed a theory where the large scale phenomena of the biosphere could be understood in terms of tiny component parts; a similar challenge that he faced in explaining large scale phenomena of thermodynamics in terms of the motion of atoms and molecules.

He was also, I believe, the first to view Darwin’s selection principle in terms of thermodynamic entropy, anticipating Schrodinger great insights by sixty years (7).

The general struggle for existence of living beings is therefore not a fight for the elements – the elements of all organisms are available in abundance in air, water, and soil – nor for energy, which is plentiful in the form of heat, unfortunately untransformably, in every body. Rather it is a struggle for entropy that becomes available through the flow of energy from the hot Sun to the cold Earth. To make the fullest use of this energy, the plants spread out the immeasurable areas of their leaves and harness the Sun’s energy by a process that is still unexplored, before it sinks down to the temperature level of the Earth, to drive chemical syntheses of which one has no inkling as yet in our laboratories. The products of this chemical kitchen are the object of the struggle in the animal world

Boltzmann felt harassed by philosophers such as Mach who persisted in scoffing at atomic theory and he viewed their behaviour as a result of the conservative nature of cultural evolution. With surprisingly modern insight, anticipating current theories of cultural evolution such as Gintis' and memetics, he saw that beliefs, especially philosophical beliefs, are transmitted through a Darwinian process (8).

Kant’s antinomies and other ‘laws of thought’ are subject to Darwinian evolution, hence they change and can be inherited.

With this insight in hand he seems understanding, perhaps even forgiving, of the philosophers whose intransigence would eventually contribute to his suicide. He understood (8):

that people inherited bad philosophy from the past and that it was hard for scientists to overcome such inheritance.

Boltzmann also understood the deep insight that not just philosophical ideas but also scientific ideas or theories were subject to Darwinian evolution. He especially took exception to the immutability of ‘self-evident’ concepts (9):

Boltzmann gives examples where these laws have been falsified by empirical knowledge: one example is the geocentric theory, with its absolute conception of the antipodes, etc. Such conceptions ‘at the time were regarded as self-evident laws of thought, whereas we are now convinced that they are futile’.

Thus Boltzmann viewed scientific theories as involved in a Darwinian process where successful theories are selected on the basis of the evidence and thereby anticipated Donald Campbell and Karl Popper by fifty years.

Figure 13: Ludwig Boltzmann, a founder of statistical mechanics, believed in both Darwinian cultural and scientific evolution.

The theory of statistical mechanics he developed takes this to a whole other level. When considering a volume of gas composed of atoms, for example helium gas, he considered the model of all the states of position and momentum each atom could have. He was able to show that if we know the total energy of the gas by measuring its temperature then it is it exceedingly likely that the gas will be in a state where each of the vast number of atoms has roughly equal energy. This is merely due to the assumption that each possible state has equal probability and that there are far more possible states of roughly equal energy than there are states where the energy is concentrated in specific groups of atoms.

This theory involves an immense hypothesis space where each hypothesis is initially assigned equal probability. As we gain empirical evidence of macroscopic variables of such as a temperature or pressure we consider these as constraints defining a remaining family of possible hypotheses; the hypotheses space is reduced but each possibility within the reduced space is still assigned equal probability. If we could learn enough macro-variables we could theoretically reduce the hypothesis space to a single certain state (10).

Here we have a theory, not of a timeless scientific law, but for determining the transient state of a particular system. We progress towards certainty in a Darwinian/Bayesian manner of selecting the ever smaller hypothesis space as constrained by our ever greater evidence. As we will see statistical mechanics is a scientific inferential system, one which operates within the cultural institution of science. This is in contrast to biology which also describes an inferential system, but one which operates in the biosphere. 

Scientific models of biology are second order models or models of models in the sense that the scientific models describe a biological model: the genome. On the other hand the scientific model of statistical mechanics is a first order model, one that directly models reality. This last statement deserves some qualification: statistical mechanics makes inferences on a quantum description of matter. It is possible that this quantum description does not directly describe reality but rather describes a model of reality at a deeper level, in which case both the scientific theories of quantum mechanics and statistical mechanics should be considered second order theories.

In the next chapter I will present arguments supporting the view that quantum theory is in fact a model of reality at a deeper level, a model which may have much in common with that of biology.

Conclusion

I have considered the view that cultural evolution may entail a Darwinian/Bayesian process and that scientific evolution may be a relatively efficient example of this type of cultural evolution. In this view science not only evolves through this process but it has found that Darwinian/Bayesian evolutionary processes are a common fundamental property of many of the complex entities it studies.This suggests that not only the complexities of culture require this type of evolutionary mechanism but that they may be common and fundamental components of much of the complex phenomena found in nature.

Bibliography

1. A framework for the unification of the behavioral sciences. Gintis, Herbert. 2007, BEHAVIORAL AND BRAIN SCIENCES.

2. Learning Through Noticing: Theory and Experimental Evidence in Farming. Hanna, Rema , Mullainathan, Sendhil and Schwartzs, Joshua . s.l. : NBER Program(s): DEV LS PR, 2014, Vol. NBER Working Paper No. 18401.

3. Jaynes, Edwin T. Bayesian Methods: General Background. [book auth.] J. H. Justice (ed.). Maximum-Entropy and Bayesian Methods in Applied Statistics. Cambridge : Cambridge Univ. Press, 1986.

4. Wikipedia. Francais Galton. Wikipedia. [Online] [Cited: November 1, 2014.] http://en.wikipedia.org/wiki/Francis_Galton.

5. —. Ludwig Boltzman. Wikipedia. [Online] [Cited: June 7, 2014.] http://en.wikipedia.org/wiki/Ludwig_Boltzmann.

6. Hull, David L. Science as a Process: An Evolutionary Account of the Social and Conceptual Development of Science. Chicago and London : The University of Chicago Press, 1988.

7. Schuster, Peter. Boltzmann and Evolution: Some Basis Questions of Biology seen with Atomistic Glasses. [book auth.] Giovanni Gallavotti, Wolfgang L. Reiter and Jakob Yngvason. Boltzmann's Leggacy. s.l. : European Mathematical Society, 2008.

8. Blackmore, John. Ludwig Boltzmann: His Later Life and Philosophy, 1900 - 1906. s.l. : Springer, 1995.

9. D'Agostino, S. A History of the Ideas of Theoretical Physics. s.l. : Springer, 2001.

10. Gibbs vs Boltzmann Entropies. Jaynes, E.T. 1965, Am. J. Phys., 33, 391.