John O. Campbell
In May I made a post titled 'Statistical Mechanics' which is an excerpt from my book Darwin Does Physics. It has proven to be the most popular post on this blog so I decided to post a further excerpt from the book that may provide context.
A Unified theory of Behavioural Sciences
The behavioural sciences, which attempts to explain and
describe the processes of human and animal behaviour from many different
perspectives, contain a plethora of evolutionary theories. These various
perspectives have generated a tower of babel of overlapping jargon and
concepts. Many employ a Darwinian/Bayesian approach.
Some behavioural scientists, such as the economist Herbert
Gintis, have proposed a single Darwinian/Bayesian theory to encompass the many
different schools of the behavioural sciences (1). Gintis calls his
theory the Beliefs, Preferences, and Constraints (BPC) model and bases it on
two basic processes: Darwinian evolution and Bayesian game theory.
Gintis acknowledges the reliance which both neural and
cultural mechanism have on biological mechanisms resulting in the view that
cultural is a nested hierarchy of evolutionary processes. He draws upon the gene-culture
co-evolution model of cultural evolution which explains the interplay of
cultural and biological mechanisms within this arena.
Figure 11: Economist Herbert
Gintis is one of a number of Social Scientist to propose a unified theory of
cultural evolution.
Darwinian component
The BPC model views social learning between generations or
the process of socialization as the dominant Darwinian process of cultural
evolution. It is a process employed by all cultures and has the hallmarks
of a Darwinian process: there is copying of behaviours and beliefs between
generations, there is variation amongst these copied beliefs and behaviours,
and there is selection.
Cultural transmission generally takes the form of conformism– that is,
individuals accept the dominant cultural forms, ostensibly because it is
fitness-enhancing to do so. Although adopting the beliefs, techniques, and
cultural practices of successful individuals is a major mechanism of cultural transmission,
there is constant cultural mutation, and individuals may adopt new cultural
forms when those forms appear better to serve their interests.
Bayesian Component
The other pillar of Gintis’ theory is Bayesian game theory
which Gintis views as the selection mechanism for his evolutionary theory.
The analysis of living systems includes one concept that does not occur
in the nonliving world and that is not analytically represented in the natural
sciences. This is the notion of a strategic interaction, in which the behavior of individuals is derived by assuming that each
is choosing a fitness-relevant response to
the actions of other individuals. The study of systems in which individuals
choose fitness-relevant responses and in which such responses evolve
dynamically, is called evolutionary game theory.
In this view individuals and cultures jockey for Darwinian
success on the basis of their preferences and beliefs. While our beliefs may be
highly uncertain or inaccurate, evolutionary game theory allows us to do the
best we can with the beliefs we have and maybe even to occasionally update our
beliefs. The Bayesian nature of Gintis’ Belief, Preferences and Constraints
theory is reflected in his treatment of Beliefs.
It follows that beliefs are the underdeveloped member of the BPC
trilogy. Except for Bayes’ rule, there is no compelling analytical theory of
how a rational agent acquires and updates beliefs, although there are many
partial theories
Thus in the terms of this theory cultural evolution is a
Darwinian process where selection is made amongst human variations in beliefs
and preferences on the basis of the consequences these have. It is a
fundamentally conservative process in that we are loathe to change beliefs and
preferences inherited from the previous generation.
Because cultural beliefs tend to be inherited and their
relationship to the evidence is not examined too carefully, cultural evolution
may seem excruciatingly slow. Much of this inertia may be due to the faith we
place in those social beliefs we learned as children. Faith is an inherently
illogical component of cultural evolution for faith implies a commitment to
belief regardless of the evidence at hand. While this kind of conservative
strategy may make sense in many situations it is illogical and inefficient if
we have sufficient evidence to accurately assess the beliefs we have inherited.
In many settings a conservative approach makes good sense.
If our ancestors have passed down a set of behaviours with proven survival
ability it would be unwise to make random changes.
Although it is generally inefficient and conservative, culture
acts as an inferential system where models based on beliefs are updated on the
basis of their experience in the world. There are time lags, often on the order of
many generations, between new evidence and the changing of beliefs. Often the
updating of beliefs occurs only on the basis of survival, when one culture
replaces another.
A simple example in which the evolution of a cultural
processes is inefficient is supplied by a study of traditional Indonesian
seaweed farmers. The study concluded that over years of experience the farmers
had found the near optimal spacing of seaweed pods but they had failed to find
the optimal size for the pods (2). In Bayesian terms,
the evolution of their farming technique was sub-optimal because it failed to
gather evidence on an exhaustive hypothesis space.
It is highly plausible that a theory resembling Gintis’ will
eventually succeed in providing a comprehensive explanation of cultural
evolution. Possibly human cultures will become more adept at updating
themselves in an efficient Bayesian manner. The evolution of science provides
an example of a cultural process whose efficiency as a inferential system has
evolved over time.
Perhaps science is the cultural process with most freedom from
the inertia of faith. While scientists are often loathe to abandon their beliefs,
the scientific method dictates that gut-feelings, intuitions or authoritative
ancient texts carry little weight in the face of clear evidence; beliefs must
be changed to conform with the evidence.
While the concept that hypotheses must be selected on the basis
of evidence has been fundamental to science since Bacon the implication was
unclear, at first, that scientific hypothesis are uncertain and are best
described with probabilities. Understanding the role of probability and
Bayesian inference within science is a central theme of its history. Pascal and
Fermat introduced some probabilistic ideas to mathematics during Newton’s
lifetime but only for the purpose of analyzing gambling games. In 1713, eight
years after Jacob Bernoulli’s death, his great work on probability was published
which extended the mathematics of probability and suggested some scientific
applications. Probability only became part of the scientific tool box in 1774
when Laplace independently discovered Bayes theorem and used it to solve some
outstanding scientific problems such as the ‘great Jupiter–Saturn inequality’ (3).
However even Laplace used Bayesian inference merely to
resolve ambiguities in data rather than as an essential element of the
scientific method. It was not until the mid-nineteenth century that James
Maxwell introduced probabilities as a fundamental concept in science when he proposed
an equation which described the velocity of gas molecules at a given
temperature in terms of a probability distribution. His work was extended by
Boltzmann and Gibbs into the field of statistical mechanics which was able to
derive, using what turned out to be Bayesian inference, the many findings of
thermodynamics from the kinetic or atomic theory of materials.
Beginning in the 1870s Francais Galton introduced
probabilistic and statistical methods for the purpose of analyzing biological
variations. His methods were later developed by Karl Pearson and R.A. Fisher,
largely in a biological context, within the frequentest school of probability.
Unfortunately Galton, who was the cousin of Charles Darwin, was-over
enthusiastic in his understanding of the power which genetics has for molding
humans. He tended to believe that the human race might be improved if its
fittest members were encouraged to have a greater number of children. He coined
the word ‘Eugenics’ which developed into a barbarous cultural influence responsible
for, amongst other atrocities, the castration of low IQ people who fell into
the hands of the authorities in many countries including Belgium, Brazil, Canada, Japan and Sweden. The
study of Eugenics may also have supported notions of racial superiority which were
central to some of the worst forms of 20th century ideology including
Nazism.
Although Eugenics developed into a monstrous cultural
practice, Galton’s understanding of Eugenics was much more restrained and
humane (4):
Galton invented the term eugenics
in 1883 and set down many of his observations and conclusions in a book, Inquiries
into Human Faculty and Its Development. He believed that a
scheme of 'marks' for family merit should be defined, and early marriage
between families of high rank be encouraged by provision of monetary
incentives. He pointed out some of the tendencies in British society, such as
the late marriages of eminent people, and the paucity of their children, which
he thought were dysgenic.
He advocated encouraging eugenic marriages by supplying able couples with
incentives to have children.
However, the ugly racial scar to which the development of
Eugenics contributed should not negate the central contributions which Galton
made to the development of biology. By introducing probability and statistics
to the study of biological systems he provided the central tools which would be
used in constructing biological models. Thus he should be considered one of the
founders of modern biological science, in particular the field of bio-metrics.
Figure 12: Francais Galton
introduce probability and statistics to the study of biology.
Later in the first half of the twentieth century probability
was found to be a core component of the quantum description of matter.
In this manner physics and biology underwent revolutions with
the introduction of probabilities and models were introduced. Although it was
not immediately clear, each of these innovations can be understood as examples
of inferential systems. It is interesting that at the early stages of their
discovery each of these inferential systems were widely thought to be only
abstractions or methods of calculation; they were not thought to have an actual
existence in reality.
This is still the consensus view of quantum theory but it
may be surprising that atomic theory was also in a similar state of limbo for
nearly a hundred years. During the nineteenth century the physics establishment
rejected the idea that atoms could actually exist and many did not believe that
any direct experimental evidence would ever be found for anything so small as
atoms. This was inconvenient for Ludwig Boltzmann who was attempting to derive
thermodynamic results by modeling materials as large collections of atoms (5):
He had a long-running dispute with the editor of the preeminent German
physics journal of his day, who refused to let Boltzmann refer to atoms and
molecules as anything other than convenient theoretical constructs. Only a couple of years after Boltzmann's death, Perrin's studies of colloidal suspensions (1908–1909), based on Einstein's theoretical
studies of 1905, confirmed
the values of Avogadro's
number and Boltzmann's
constant, and convinced
the world that the tiny particles really exist.
At about the same time that Boltzmann faced disbelief and
scorn over the existence of atoms Gregor Mendel published a little noticed
paper which described his experimental results in pollinating and growing 5,000
pea plants. His noted a law in his results; when a plant having a dominant
characteristic, say a purple flower, is crossed with a plant having the
recessive characteristic, say a white flower, their offspring will all produce
purple flowers, the dominant characteristic. If these second generation purple
flowered plant are then crossed their offspring will produce both purple and
white flowers in a ratio of 3:1.
Mendel referred to an inherited ‘factor’ which distinguished
between purple and white flowers in each generation. In modern terminology his
term ‘factor’ has been replaced with ‘gene’. At the time his findings were thought
both limited and unimportant and were ignored for several decades. However the
problem of inheritance was one of the main gaps in the developing Darwinian
theory of evolution. Many biologist assumed that inherited traits would tend to
blend, that crossing purple and white flowered pea plants should result in
light purple offspring. Darwin himself favoured a theory of ‘pangenesis’ which failed to hold up to experimental examination.
In their search for clues to the nature of biological
inheritance some researchers re-discovered Mendel’s paper and were able to
reproduce his experiments. Mendel’s long forgotten work rapidly became a
center-piece of biological theory.
A heated debate then occurred as to the compatibility of
Mendel’s and Darwin’s theories. Whether a leading biologist did or did not
believe that Mendel’s theory was compatible with Darwin’s they shared a
consensus that Mendel’s factors or genes had to be understood as a
calculational device; they did not have an actual existence (6).
Of course both atoms and genes are now understood to
actually exist but we might wonder about the heated initial resistance to these
concepts. I suggest that one reason for the disbelief is that Boltzmann and
Mendel assigned probabilities to atoms and genes. Probabilities meant
uncertainty which for science, dominated as it was by the certain clockwork
mechanisms of the Newtonian paradigm, was almost blasphemous. Much better that theories
involving probabilities should be considered mere approximations or aids to calculations
until the real deterministic theory is found.
We have also seen that where there are probabilities there
is also information, entropy and an internal model; the existence of a
probability implies an entire inferential system. This was a lot for science to bite off
especially at the beginning when only the tip of the ice berg was yet discovered.
Biology has successfully made the transition. The Neo-Darwinism model of
biology is the scientific consensus and clearly describes an inferential system
independent of human calculation.
Boltzmann
Perhaps Boltzmann’s story and the development of statistical
mechanics based on atomic theory is even more interesting. Surprisingly
Boltzmann was a big fan of Darwin (7).
If you ask me about my innermost conviction whether our century will be
called the century of iron or the century of steam or electricity, I answer
without hesitation: It will be called the century of the mechanical view of
Nature, the century of Darwin.
Boltzmann understood that Darwin had constructed a theory where
the large scale phenomena of the biosphere could be understood in terms of tiny
component parts; a similar challenge that he faced in explaining large scale
phenomena of thermodynamics in terms of the motion of atoms and molecules.
He was also, I believe, the first to view Darwin’s selection
principle in terms of thermodynamic entropy, anticipating Schrodinger great insights
by sixty years (7).
The general struggle for existence of living beings is therefore not a fight for the elements – the elements of all organisms are available in abundance in air, water, and soil – nor for energy, which is plentiful in the form of heat, unfortunately untransformably, in every body. Rather it is a struggle for entropy that becomes available through the flow of energy from the hot Sun to the cold Earth. To make the fullest use of this energy, the plants spread out the immeasurable areas of their leaves and harness the Sun’s energy by a process that is still unexplored, before it sinks down to the temperature level of the Earth, to drive chemical syntheses of which one has no inkling as yet in our laboratories. The products of this chemical kitchen are the object of the struggle in the animal world
Boltzmann felt harassed by philosophers such as Mach who
persisted in scoffing at atomic theory and he viewed their behaviour as a
result of the conservative nature of cultural evolution. With surprisingly
modern insight, anticipating current theories of cultural evolution such as
Gintis' and memetics, he saw that beliefs, especially philosophical beliefs, are
transmitted through a Darwinian process (8).
Kant’s antinomies and other ‘laws of thought’ are subject to Darwinian
evolution, hence they change and can be inherited.
With this insight in hand he seems understanding, perhaps
even forgiving, of the philosophers whose intransigence would eventually
contribute to his suicide. He understood (8):
that people inherited bad philosophy from the past and that it was hard
for scientists to overcome such inheritance.
Boltzmann also understood the deep insight that not just philosophical
ideas but also scientific ideas or theories were subject to Darwinian
evolution. He especially took exception to the immutability of ‘self-evident’ concepts (9):
Boltzmann gives examples where these laws have been falsified by
empirical knowledge: one example is the geocentric theory, with its absolute
conception of the antipodes, etc. Such conceptions ‘at the time were regarded
as self-evident laws of thought, whereas we are now convinced that they are
futile’.
Thus Boltzmann viewed scientific theories as involved in a
Darwinian process where successful theories are selected on the basis of the
evidence and thereby anticipated Donald Campbell and Karl Popper by fifty years.
Figure 13: Ludwig Boltzmann, a
founder of statistical mechanics, believed in both Darwinian cultural and
scientific evolution.
The theory of statistical mechanics he developed takes this
to a whole other level. When considering a volume of gas composed of atoms, for
example helium gas, he considered the model of all the states of position and
momentum each atom could have. He was able to show that if we know the total
energy of the gas by measuring its temperature then it is it exceedingly likely
that the gas will be in a state where each of the vast number of atoms has
roughly equal energy. This is merely due to the assumption that each possible
state has equal probability and that there are far more possible states of
roughly equal energy than there are states where the energy is concentrated in
specific groups of atoms.
This theory involves an immense hypothesis space where each
hypothesis is initially assigned equal probability. As we gain empirical
evidence of macroscopic variables of such as a temperature or pressure we
consider these as constraints defining a remaining family of possible
hypotheses; the hypotheses space is reduced but each possibility within the
reduced space is still assigned equal probability. If we could learn enough
macro-variables we could theoretically reduce the hypothesis space to a single
certain state (10).
Here we have a theory, not of a timeless scientific law, but
for determining the transient state of a particular system. We progress towards
certainty in a Darwinian/Bayesian manner of selecting the ever smaller
hypothesis space as constrained by our ever greater evidence. As we will see
statistical mechanics is a scientific inferential system, one which operates
within the cultural institution of science. This is in contrast to biology
which also describes an inferential system, but one which operates in the biosphere.
Scientific models of biology are second order models or
models of models in the sense that the scientific models describe a biological model: the genome. On the other hand the scientific model of statistical mechanics is a first
order model, one that directly models reality. This last statement deserves
some qualification: statistical mechanics makes inferences on a quantum
description of matter. It is possible that this quantum description does not directly
describe reality but rather describes a model of reality at a deeper level, in which
case both the scientific theories of quantum mechanics and statistical
mechanics should be considered second order theories.
In the next chapter I will present arguments supporting the
view that quantum theory is in fact a model of reality at a deeper level, a
model which may have much in common with that of biology.
Conclusion
I have considered the view that cultural evolution may entail a Darwinian/Bayesian process and that scientific evolution may be a relatively efficient example of this type of cultural evolution. In this view science not only evolves through this process but it has found that Darwinian/Bayesian evolutionary processes are a common fundamental property of many of the complex entities it studies.This suggests that not only the complexities of culture require this type of evolutionary mechanism but that they may be common and fundamental components of much of the complex phenomena found in nature.
Bibliography
1. A framework for the
unification of the behavioral sciences. Gintis, Herbert. 2007,
BEHAVIORAL AND BRAIN SCIENCES.
2. Learning Through Noticing: Theory and Experimental
Evidence in Farming. Hanna, Rema , Mullainathan, Sendhil and
Schwartzs, Joshua . s.l. : NBER Program(s): DEV LS PR, 2014, Vol.
NBER Working Paper No. 18401.
3. Jaynes, Edwin T. Bayesian Methods: General
Background. [book auth.] J. H. Justice (ed.). Maximum-Entropy and Bayesian
Methods in Applied Statistics. Cambridge : Cambridge Univ. Press,
1986.
4. Wikipedia. Francais Galton. Wikipedia. [Online]
[Cited: November 1, 2014.] http://en.wikipedia.org/wiki/Francis_Galton.
5. —. Ludwig Boltzman. Wikipedia. [Online] [Cited:
June 7, 2014.] http://en.wikipedia.org/wiki/Ludwig_Boltzmann.
6. Hull, David L. Science as a Process: An
Evolutionary Account of the Social and Conceptual Development of Science. Chicago
and London : The University of Chicago Press, 1988.
7. Schuster, Peter. Boltzmann and Evolution: Some
Basis Questions of Biology seen with Atomistic Glasses. [book auth.] Giovanni
Gallavotti, Wolfgang L. Reiter and Jakob Yngvason. Boltzmann's Leggacy. s.l. :
European Mathematical Society, 2008.
8. Blackmore, John. Ludwig Boltzmann: His Later
Life and Philosophy, 1900 - 1906. s.l. : Springer, 1995.
9. D'Agostino, S. A History of the Ideas of
Theoretical Physics. s.l. : Springer, 2001.
10. Gibbs vs Boltzmann Entropies. Jaynes, E.T.
1965, Am. J. Phys., 33, 391.
