Thursday, 21 November 2019

Inferential Systems and the Causal Revolution

John O. Campbell

This is an excerpt from the upcoming book: The Knowing Universe.

As we have discussed, an important aspect of inferential systems is their autopoietic form; inferential systems cause entities to exist. That is, the knowledge which models accumulate is causal knowledge capable of causing the specific structures, adaptations and regulation which form the entity and allow it to resist natural forces towards dissipation.

Unfortunately, scientist and statisticians often consider ‘cause’ as something of a forbidden concept because data or evidence, on its own, can indicate a correlation but not a causal relationship. An oft quoted example is that the rising sun and the crowing rooster are correlated but that the rooster’s crowing does not cause the sun’s rising. Something beyond data is required to establish causality.
A recent scientific innovation led by Judea Pearl, sometimes called the ‘causal revolution’ (128; 129), may have completed the logic of science as described by Bayesian probability (61). In particular, the causal revolution provides some important new understanding that illuminates the autopoietic nature of inferential systems underlying existence.

While the causal revolution has provided methods of understanding new to science, these methods may have previously been unconsciously discovered by humans in the course of human genetic evolution. It is widely understood that sometime between 70,000 and 30,000 years ago a series of mutations in the genes specifying our neural structures resulted in a ‘cognitive revolution’ that powered our species cultural evolution and provided key abilities enabling our (perhaps temporary) ascendency over other biological forms. Some researchers attribute this cognitive revolution to a subconscious realization of casual reasoning, the same causal reasoning that is now becoming consciously understood and that forms the basis of the scientific causal revolution (130; 129; 131).
In this view the current scientific revolution is a mere rediscovery, in conscious terms, of a fitness enhancing reasoning process which natural selection placed in our subconscious minds, tens of thousands of years ago. 

Unfortunately, so far, the causal revolution has acknowledged the process of causal reasoning to be an ability possessed only by our species. In this section we will demonstrate that causal reasoning, like the more general process of Bayesian inference, is a key component of the process of evolutionary change which has brought into existence and evolved reality’s many domains including the cosmological, quantum, biological, neural and cultural domains (13)

The causal revolution is better cast as a recent scientific discovery of a general law of nature which underlies the many existing forms composing our universe and which has operated since the beginning, long before the evolution of our species. In that sense the causal revolution provides powerful tools for understanding inferential systems and general evolutionary processes as described by the theory of universal Darwinism (4; 6; 13; 9).

We will not only demonstrate that the relationship between causal reasoning and inferential systems provides a generalized context in which causal reasoning is found to participate in the universe’s many evolutionary processes but will also provide specific details of causal mechanisms operating in inferential systems.  The causal revolution understood within the context of autopoietic inferential systems provides insights into the widespread role of causality within natural systems.
Contrary to traditional statistics and the mantra of the ‘big data’ movement that ‘data is everything and everything is data’[1], Pearl points out that this perspective severely constrains the scope of statistics. Traditional statistics only views data as describing correlations. Correlations are not causes as illustrated by the cliché of the crowing rooster and the rising sun.

Pearl demonstrates that descriptions of causal relationships require an ingredient beyond data, in addition they require a model that hypothesizes causal relationships. As Pearl describes it (129):

The model should depict, however qualitatively, the process that generates the data – in other words, the cause and effect forces that operate in the environment and shape the data generated.

The generative model described by Pearl is the same model that operates within inferential systems, a model composed of a competing family of hypotheses that explain how the evidence was generated or caused. Thus, effective models are constrained to model how existing entities and processes are generated or brought into existence.

Pearl’s preferred type of causal model is the causal diagram which depicts the variables involved in a process as points and connects these points with arrows that serve to hypothesize their cause and effect relationships. He is clear that these causal diagrams are hypotheses or best guesses as to the actual causal relationship and that they must be subject to the scrutiny of the data or evidence.
If the evidence does not support the diagram, Pearl suggest we should take another, perhaps informed, guess as to the actual relationship, and test its implications against the data. We should continue with these guesses until a diagram is discovered that is consistent with all the data (129):

If the data contradict this implication, then we need to revise our model.

Rather than test one hypothesis at a time, a more systematic approach often seen with inferential systems, is to consider a model composed of a complete family of competing hypotheses describing the causal connections between the variables. Then application of the data to the model consists of updating the probability assigned to each possible hypothesis. Importantly this type of model often quickly simplifies as the data eliminates unsupported hypotheses.

We might note Pearl’s reference above to ‘cause and effect forces’ that are described by causal models. As noted in previous sections, system regulators generically overcome challenges to existence posed by physical law through the application of cause and effect forces. It is in this sense that the knowledge of inferential systems initiates a causal chain of forces which form and maintain systems within existence, and it is in this sense that inferential systems are autopoietic.
It is important to understand what Pearl means by causation:

For the purpose of constructing the diagram, the definition of ‘causation’ is simple, if a little metaphorical: a variable X is a cause of Y if Y ‘listens’ to X and determines its value in response to what it hears.

Although Pearl’s description is both clear and accurate, the term ‘listens to’ may be a little anthropomorphic when applied to abstract variables. Perhaps a better term is ‘receives information from’ and then his definition of causation becomes:

A variable X is a cause of Y if Y receives information from X and determines its value in response to that information.

In these terms, causation, as defined by Pearl, is an exact analog to inferential systems; models within inferential systems are updated by information or evidence in the sense that the value of the probability of each hypothesis composing the model is updated in response to the information or evidence it receives. This update or response has been generally described in terms of force; the evidence may be said to ‘force’ some of a model’s hypotheses to be assigned greater probability and some less (21). Thus, the model is shaped by the force of the evidence and we may say that the evidence causes the resulting model.

This analogy between causation and force is widely accepted. As Wikipedia tells us (132):

Causal relationships may be understood as a transfer of force. If A causes B, then A must transmit a force (or causal power) to B which results in the effect. 

In these terms, considering the close analogy between causation, inference and force, inferential systems may be considered the fundament form of causation. It is forces which cause effects and forces may always be understood in terms of the updating of inferential systems. This generic concept of force has been explored by Steven Frank (born 1957) who concludes that the process of inference may be considered as the force of data applied to models (21)

As we have seen in our previous discussion of physical forces, at the fundamental physical level a force is produced when information contained in a gauge boson updates the wave function of another quantum system. In Pearl’s terms we might say that the quantum system listens to the gauge boson and determines or updates its value, for example the value of its momentum, in response to what it hears.

Although nature has employed inferential systems since the beginning as its primary engine of existence, scientific understanding is only now becoming able to more fully describe them. The causal revolution provides our scientific understanding with important tools for describing the autopoietic aspect of inferential systems. In short, this new tool allows us to scientifically describe inferential systems as initiating a cascade of causes whose effects are existing systems. The mechanisms by which causes lead to effects may be understood in terms of orchestrated or regulated forces and these forces, as we have seen, are essential to overcome the many natural obstacles to existence.  The initiated causes must be highly orchestrated to achieve the effect of existence and knowledge required to achieve this orchestration is accumulated by the inferential system through the process of Darwinian evolution.  

A breakthrough made by the causal revolution is its understanding that a full scientific description of a causal process must include a causal model of the hypothesised causal cascade. Central to our understanding of inferential systems is that this same causal model is necessary to orchestrate or regulate existing systems as required by the good regulator theorem. In other words, the causal revolution has discovered that scientific descriptions of existing systems must include a causal model, and this is necessary because causal models are an essential component of existing systems; a scientific description of them that did not include a causal model would be incomplete. 

Central to the causal revolution’s understanding is that causation often involves interventions made to the normal or spontaneous unfolding of events. If an effect is caused, that means that the normal range of possible outcomes is constrained to a single outcome and this selection is described as occurring through an intervention in the normal course of events (128). In terms of inference, circumstances are causally orchestrated so that the received data updates the model to strongly predict or initiate a single outcome or effect.

At the core of autopoietic inferential systems is their ability to cause outcomes, to intervene in the spontaneous course of events and select a single outcome that might otherwise be extremely unlikely. As a biological example, we might consider that the knowledge contained in an organism’s genome causes specific outcomes or effects. A given three letter genetic codon identifies a single specific amino acid to be added to a protein and a gene, composed of a string of codons, identifies a specific complete protein. In turn, that protein, if it is an enzyme, may catalyze a specific bio-chemical reaction, an outcome or effect that is extremely unlikely to occur without the participation of the enzyme. This biological cascade of knowledgeable causes, resulting in specific effects, illustrates the tight causal control or regulation exercised by autopoietic inferential systems.

Pearl describes the causal revolution in terms of a ladder having three rungs which together fully describe causal systems. The first rung is characterized by correlations, the traditional study of statistics. The second rung is characterized by causal interventions of the type we have just examined. The third rung involves counterfactuals or hypothesis regarding what might be rather than what is. A dictionary example of a counterfactual hypothesis is ‘If kangaroos had no tails, they would topple over (133).

While causal interventions describe the autopoietic aspect of inferential systems in that they are required to ensure that the system’s knowledge causes a specific process of self-creation and self maintenance, counterfactual hypotheses describe the evolutionary aspects of inferential systems as they make hypotheses which have never been tested before. Counterfactual hypotheses are tools which may be used to search through the space of possibilities, a search for those possibilities which may be made actual, which may be brought into existence. 

For example, the counterfactual hypothesis involving kangaroos and their tails may be coded in the genome of a kangaroo where a mutational cause produces the effect of a tailless kangaroo offspring. The mutant gene codes the counterfactual hypothesis which in effect asks if a kangaroo with no tail would be reproductively successful (didn’t always fall over). If the resulting tailless kangaroo did achieve reproductive success the possibility would be made actual and tail-less kangaroos would achieve existence in the world.

The causal revolution has extended scientific understanding of causation beyond correlations to include causal models, interventions and counterfactual hypotheses. I have provided examples from biology to illustrate how this extended understanding describes actual causal processes central to biology and in chapter 3 we will see that this understanding is also central to domains of reality other than biology. In effect autopoietic inferential systems in all domains cause their own existence and evolve through their exploration of counterfactual hypotheses.

However, it appears that those who forged the causal revolution consider it more of a revolution in scientific calculation than in the understanding of actual phenomena found in nature. Pearl, for example writes extensively on how causal models can assist the computation of relationships between variables but nowhere does he suggest that these models actually exist in the natural world or their essential role in bringing actual phenomena into existence (129). Instead he understands the findings of the causal revolution as calculational devices that allow us to calculate data and arrive at causal conclusions.

This is a common misunderstanding when a scientific revolution first brings some important new physical phenomena to light for which there is little direct observational evidence. It occurred, for example, in the late 1800s when many considered the controversial concept of atoms as referring to a shorthand for making scientific calculations rather than to actual phenomena. When Boltzmann tried to publish his work deriving thermodynamics from atomic theory, his journal editors insisted that he refer to atoms as ‘bilder’; merely models or pictures that were not ‘real’ (134). Again, in the early 1900s when Mendel’s concept of genes was rediscovered there ensued a great debate among biologist on whether Mendel’s theory conflicted with Darwin’s. The one thing agreed upon by most leading biologists was that Mendel’s ‘genetics’ was not a physical process but merely a means of making calculations. As recounted by the philosopher of science David Hull (37):

As much as Bateson might disagree with Pearson and Weldon about the value of Mendelian genetics, he agreed with them that it was unscientific to postulate the existence of genes as material bodies. They were merely calculation devices.

As in the case of atoms and genetics I am confident that history will demonstrate that the new calculational methods of the causal revolution describe actual physical reality. The task of science is to describe nature and when new calculational tricks are discovered that provide powerful methods of describing reality at a deeper level, we may rest assured this is because nature also uses those same tricks to create and maintain the actual phenomena, in other words, that science has merely rediscovered and described nature’s own methods. This is the proper role of science.


1. Causal inference in statistics: an overview. Pearl, Judea. s.l. : Statistics Surveys, 2009, Vols. Volume 3 (2009), 96-146.

2. Pearl, Judea and Mackenzie, Dana. The Book of Why: The New Science of Cause and Effect. s.l. : Basic Books, 2018. ISBN-10: 046509760X.

3. Jaynes, Edwin T. Probability Theory: The Logic of Science. s.l. : University of Cambridge Press, 2003.

4. Harari, Yuval Noah. Sapiens: A brief history of humankind. s.l. : Harvill Secker, 2014.

5. Boyer, Pascal. Minds make societies: How Cognition Explains the World Humans Create. s.l. : Yale University Press, 2018.

6. Universal Darwinism as a process of Bayesian inference. Campbell, John O. s.l. : Front. Syst. Neurosci., 2016, System Neuroscience. doi: 10.3389/fnsys.2016.00049.

7. Dennett, Daniel C. Darwin's Dangerous Idea. New York : Touchstone Publishing, 1995.

8. Blackmore, Susan. The Meme Machine. Oxford, UK : Oxford University Press, 1999.

9. Bayesian Methods and Universal Darwinism. Campbell, John O. s.l. : AIP Conference Proceedings, 2009. BAYESIAN INFERENCE AND MAXIMUM ENTROPY METHODS IN SCIENCE AND ENGINEERING: The 29th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering. AIP Conference Proceedings, Volume 1193. pp. 40-47.

10. Simple unity among the fundamental equations of science. Frank, Steven A. s.l. : arXiv preprint, 2019.

11. Wikipedia. Causal reasoning. Wikipedia. [Online] [Cited: May 26, 2019.]

12. Google dictionary. Counterfactual. Google dictionary. [Online] [Cited: June 2, 2019.]

13. Wikipedia. Ludwig Boltzman. Wikipedia. [Online] [Cited: June 7, 2014.]

14. Hull, David L. Science as a Process: An Evolutionary Account of the Social and Conceptual Development of Science. Chicago and London : The University of Chicago Press, 1988.

[1] If in doubt as to the widespread use of this meme, try googling it.

Saturday, 4 May 2019

Science And The Mind Of God

John O. Campbell

This is an excerpt from an upcoming book: The Knowing Universe.

Religious belief is a defining characteristic of our species and the cultures that we have formed, including those at the hunter-gatherer level (1). Anthropologists generally understand that the existence of these kinds of universal characteristics imply that they are grounded in the human genome. The evolutionary history of these genetically based characteristics is studied by evolutionary psychologists.  As some leading biological anthropologists have noted (2):

Recent studies of the evolution of religion have revealed the cognitive underpinnings of belief in supernatural agents, the role of ritual in promoting cooperation, and the contribution of morally punishing high gods to the growth and stabilization of human society. The universality of religion across human society points to a deep evolutionary past. 

In other words, the underpinnings of religion have evolved through the process of natural selection for at least hundreds of thousands of years. We must ask, ‘did natural selection get this right, is the religious perspective useful in the sense of conferring greater fitness?’ After all fitness is the only ‘goal’ of a Darwinian process. The quote above notes a couple of avenues of research pursued by anthropologists and evolutionary psychologists suggesting indirect forms of fitness that religion may confer such as promoting cultural cooperation and the enforcement of moral stability.  

However, these may only be by-products of religious belief, what about the content of religious beliefs, our predisposition to believe in supernatural agents? Did natural selection get that right or does our predisposition for religion only confer fitness through indirect means?

It is tempting, in this scientific age, to dismiss belief in supernatural agents as clearly mistaken and incapable of providing fitness. In fact, science is based upon naturalism, the belief that only natural processes take place in the universe (3). In this view ‘supernatural’ is an oxymoron as everything is part of nature and nothing is outside of it.

But, before dismissing the existence of supernatural agents out of hand we might consider that natural selection very rarely gets things wrong: it is a powerful method of inference (4; 5) which almost always gets things right. As Daniel Dennett wrote (6):

Getting it right, not making mistakes, has been of paramount importance to every living thing on this planet for more than three billion years, and so these organisms have evolved thousands of different ways of finding out about the world they live in

How can this conundrum be solved, how can belief in supernatural agents be consistent with our scientific understanding? The solution suggested here is that natural selection has provided us with the propensity to believe not only in supernatural agents but also in natural agents. Supernatural agents are an oxymoron in the sense that if one were discovered it would then be part of nature so instead, we can think in terms of natural agents or agents who operate within the laws of nature.
In any case, our natural propensity towards belief in powerful agents acting in the world is woefully lacking in specifics and we have little easily available evidence as to these agents’ identities. Cultures have imagined a huge variety of specific agents including ghosts, witches, spirits and Gods but given a lack of evidence that could select among the possibilities, our imaginations are free to roam. Clearly our intuitions about agents in control of the universe do not rule out natural agents. As we will see, what evidence we do have concerning agents operating in the world is evidence of natural rather than supernatural agents.

In a religious context then, the agents which created and run this universe may be natural agents. This approach to religion has already been developed by philosophers such as Baruch Spinoza (1622 to 1677) and scientist such as Albert Einstein (1879 - 1955) who considered that God is equivalent to nature as it is understood by science. According to Spinoza’s biographer, this equivalency was the primary point of his philosophy (7):

Above all Spinoza’s God is numerically identical with nature. God is nature.

Einstein confirmed Spinoza’s vision (8):

I believe in Spinoza's God who reveals Himself in the orderly harmony of what exists,

Einstein’s awesome scientific insights told him that science provided our best route to knowing God. In fact, he thought the very purpose of science is to provide us with what he called cosmic religiosity (9):

The most beautiful thing we can experience is the Mysterious — the knowledge of the existence of something unfathomable to us, the manifestation of the most profound reason coupled with the most brilliant beauty… This is the basics of cosmic religiosity, and it appears to me that the most important function of art and science is to awaken this feeling among the receptive and keep it alive.

Beliefs about God in the tradition of Spinoza and Einstein, that science can reveal God in a form equivalent to nature, is quite common among leading scientists. As Stephen Hawking put it (10):

If we do discover a theory of would be the ultimate triumph of human reason—for then we would truly know the mind of God.

Unfortunately, neither Einstein’s nor Hawking’s genius was able to describe the specifics of scientific religiosity. Einstein was hampered by the scientific concepts of his time that were in terms of clockwork mechanistic metaphors sometimes called the Newtonian paradigm. Einstein played a major role in the revolution which would sweep away Newton’s paradigm, but he was in the final years of his life before a key component of that revolution was contributed by Claude Shannon (1916 - 2001) in the form of information theory (11).

Information theory is just a perspective on the mathematics of probability theory. It focuses on a function of a probability -log2(p), where p is a probability and names this function ‘information’. Information is the surprise which an agent experiences when it assigns the probability p to some possible outcome and then discovers that outcome has actually occurred. If the agent had previously assigned a high probability to the actual outcome then there is little surprise, if it assigned a small probability then it is greatly surprised. The surprise has units of bits of information. 
For instance, if the agent initially assigned the probability 1/8 or 2-3, to what turned out to be the actual outcome, then the surprise or information experienced by the agent is 3 bits. Crucially the very concept of information presupposes an agent, mind or model which assigns a probability to something occurring in the world around it and then is surprised when it receives evidence of the actual outcome.

Shannon’s information theory raises the sceptre of agents inhabiting the world wherever information is found. The original use of Shannon’s theory was to describe electronic communication systems operated by human agents, but his theory applies equally well to any agent that can assign a probability or anticipate the likelihood of an outcome. For example, biological organisms are adapted to or anticipate a specific environment; fish expect to find themselves in water. Extreme surprise for an organism is, in this case, death. Many prominent biologists now view the core processes of biology in terms of information and information processing (12).

In general, since Einstein’s day human culture and the human conception of the world have undergone an ‘information revolution’ which has replaced the Newtonian paradigm involving clockwork industrial mechanisms with the concepts of information and information processing. The information revolution has transformed most cultural processes and is often thought of in terms of computation.

Critically, the new information metaphor has been adopted and championed by nearly every branch of scientific study. It has been found that practically all the domains of reality studied by science are readily described in terms of information theory. Quantum theory, at the basis of all physics[1], is described in terms of quantum information and quantum information processing. Genetic information is a central concept in biology. Neural based behaviour is understood in terms of brains which gather, process and act on information. And finally, human culture is understood to have emerged into an ‘Information Age’ (13).

In short, information is now a key component of all scientific understanding and some physicists understand information to be even more fundamental to our understanding of the universe than are physical entities such as mass or energy (14).  This is a radical revolution in our conception of the universe and our place in it, one that holds radical spiritual implications. I have written extensively about these implications and have suggested they be named ‘Einstein’s Enlightenment’ (15).
The information revolution allows us to scientifically consider that all of reality is inhabited by agents having their own expectations and agendas. In this view natural selection has been exonerated; the agents it has placed in our intuitions actually exist and it provides fitness by preparing us to understand reality on a new, deeper level.

On a religious level these scientific advances may allow us to better glimpse the mind of God. At a stroke we are taken from a state of great ignorance as to the specifics of religious agents, such as supernatural Gods, to a state of relative knowledge backed by hard scientific evidence. Thus, the religious enterprise becomes evolutionary; as our scientific knowledge expands, we may glimpse deeper into the mind of God.

This book will not focus on these important religious implications but will leave it to the reader to come to their own conclusion on that count. Here we will instead focus on the unifying role which the information revolution plays in portraying all areas of science in terms of a common metaphor.
I suggest a slightly different central metaphor than information as information may only exist within an ecosystem of associated concepts and processes such as probabilistic models and Bayesian inference, which we will explore later. Instead I will suggest the central metaphor of knowledge as knowledge is the output of information processing and as I will argue, the foundation of existence.
This book will develop the argument that knowledge is necessary for existence of any sort; that the laws of nature are extremely hostile to existence and existing forms must develop knowledge of autopoietic (self-creating and self-maintaining) strategies in order to overcome these challenges and to evolve new existing forms. This view may go some way in providing an answer to the obvious question ‘Why does science find information and information processing to be fundamental to all existing entities?’ The short answer it suggests is that information and information processing accumulate knowledge and knowledge is essential to all forms of existence.

Chapter 1 will review scientific understanding of the dynamic relationship between existence and knowledge. Chapter 2 will discuss the general nature of the autopoietic knowledge required to overcome the many challenges posed to existence. Chapter 3 will discuss how this general strategy to achieve existence has played out in the creation and evolution of each of the domains composing reality.

Obviously, this paradigm has deep spiritual implications which I attempted to survey in my earlier book Einstein’s Enlightenment (15):

  1.     Knowledge is the creator and sustainer of all things;
  2.   There is but one universal source of knowledge;
  3.  Humans were created by this universal source and are the most advanced form of knowledge yet found in the universe.
Rather than focusing on spiritual understanding, this book will focus on the scientific understandings, developed during the information revolution, which support this new conception of reality. Hopefully, in the tradition of Spinoza, Einstein and Hawking, we will see that spiritual and scientific understandings are much the same thing.  


1. Hunter-Gatherers and the Origins of Religion. Peoples, Hervey C., Duda, Pavel and Marlowe, Frank W. 3, Hawthorne, New York : Human Nature, Vols. 27,3. doi:10.1007/s12110-016-9260-0.
2. Temporal Naturalism. Smolin, Lee. s.l. :, 2013, Preprint.
3. Universal Darwinism as a process of Bayesian inference. Campbell, John O. s.l. : Front. Syst. Neurosci., 2016, System Neuroscience. doi: 10.3389/fnsys.2016.00049.
4. Simple unity among the fundamental equations of science. Frank, Steven A. s.l. : arXiv preprint, 2019.
5. Dennett, Daniel C. Darwin's Dangerous Idea. New York : Touchstone Publishing, 1995.
6. Nadler, Steven. Einstein's God - Prof Nadler on Spinoza, pt 1. YouTube, 2007.
7. Einstein, Albert. telegram response to New York rabbi Herbert S. Goldstein. New York : s.n., 1924.
8. —. An Ideal of Service to Our Fellow Man. NPR radio 'This I believe'.
9. Hawking, Stephen. A brief history of time: from the big bang to black holes. s.l. : Bantam Dell Publishing Group, 1988. 978-0-553-10953-5.
10. A mathematical theory of communications. Shannon, Claude. 1948, Bell System Technical Journal.
11. Theoretical bilogy in the third millenium. Brenner, Sydney. 1999, Philosophical Transactions of the Royal Society.
12. Wikipedia. Information Age. Wikipedia. [Online] [Cited: May 2, 2019.]
13. Information in the holographic universe. Bekenstein, Jacob. August 2003, Scientific American.
14. Campbell, John O. Einstein's Enlightenment. s.l. : Createspace, 2017. ASIN: B06XNZDGCS.

[1] The three forces of nature composing the standard model of particle physics are quantum forces and the fourth force found in nature, gravity, is probably emergent from quantum entanglement(189).