Thursday, 21 November 2019

Inferential Systems and the Causal Revolution

John O. Campbell

This is an excerpt from the upcoming book: The Knowing Universe.

As we have discussed, an important aspect of inferential systems is their autopoietic form; inferential systems cause entities to exist. That is, the knowledge which models accumulate is causal knowledge capable of causing the specific structures, adaptations and regulation which form the entity and allow it to resist natural forces towards dissipation.

Unfortunately, scientist and statisticians often consider ‘cause’ as something of a forbidden concept because data or evidence, on its own, can indicate a correlation but not a causal relationship. An oft quoted example is that the rising sun and the crowing rooster are correlated but that the rooster’s crowing does not cause the sun’s rising. Something beyond data is required to establish causality.
A recent scientific innovation led by Judea Pearl, sometimes called the ‘causal revolution’ (128; 129), may have completed the logic of science as described by Bayesian probability (61). In particular, the causal revolution provides some important new understanding that illuminates the autopoietic nature of inferential systems underlying existence.

While the causal revolution has provided methods of understanding new to science, these methods may have previously been unconsciously discovered by humans in the course of human genetic evolution. It is widely understood that sometime between 70,000 and 30,000 years ago a series of mutations in the genes specifying our neural structures resulted in a ‘cognitive revolution’ that powered our species cultural evolution and provided key abilities enabling our (perhaps temporary) ascendency over other biological forms. Some researchers attribute this cognitive revolution to a subconscious realization of casual reasoning, the same causal reasoning that is now becoming consciously understood and that forms the basis of the scientific causal revolution (130; 129; 131).
In this view the current scientific revolution is a mere rediscovery, in conscious terms, of a fitness enhancing reasoning process which natural selection placed in our subconscious minds, tens of thousands of years ago. 

Unfortunately, so far, the causal revolution has acknowledged the process of causal reasoning to be an ability possessed only by our species. In this section we will demonstrate that causal reasoning, like the more general process of Bayesian inference, is a key component of the process of evolutionary change which has brought into existence and evolved reality’s many domains including the cosmological, quantum, biological, neural and cultural domains (13)

The causal revolution is better cast as a recent scientific discovery of a general law of nature which underlies the many existing forms composing our universe and which has operated since the beginning, long before the evolution of our species. In that sense the causal revolution provides powerful tools for understanding inferential systems and general evolutionary processes as described by the theory of universal Darwinism (4; 6; 13; 9).

We will not only demonstrate that the relationship between causal reasoning and inferential systems provides a generalized context in which causal reasoning is found to participate in the universe’s many evolutionary processes but will also provide specific details of causal mechanisms operating in inferential systems.  The causal revolution understood within the context of autopoietic inferential systems provides insights into the widespread role of causality within natural systems.
Contrary to traditional statistics and the mantra of the ‘big data’ movement that ‘data is everything and everything is data’[1], Pearl points out that this perspective severely constrains the scope of statistics. Traditional statistics only views data as describing correlations. Correlations are not causes as illustrated by the cliché of the crowing rooster and the rising sun.

Pearl demonstrates that descriptions of causal relationships require an ingredient beyond data, in addition they require a model that hypothesizes causal relationships. As Pearl describes it (129):

The model should depict, however qualitatively, the process that generates the data – in other words, the cause and effect forces that operate in the environment and shape the data generated.

The generative model described by Pearl is the same model that operates within inferential systems, a model composed of a competing family of hypotheses that explain how the evidence was generated or caused. Thus, effective models are constrained to model how existing entities and processes are generated or brought into existence.

Pearl’s preferred type of causal model is the causal diagram which depicts the variables involved in a process as points and connects these points with arrows that serve to hypothesize their cause and effect relationships. He is clear that these causal diagrams are hypotheses or best guesses as to the actual causal relationship and that they must be subject to the scrutiny of the data or evidence.
If the evidence does not support the diagram, Pearl suggest we should take another, perhaps informed, guess as to the actual relationship, and test its implications against the data. We should continue with these guesses until a diagram is discovered that is consistent with all the data (129):

If the data contradict this implication, then we need to revise our model.

Rather than test one hypothesis at a time, a more systematic approach often seen with inferential systems, is to consider a model composed of a complete family of competing hypotheses describing the causal connections between the variables. Then application of the data to the model consists of updating the probability assigned to each possible hypothesis. Importantly this type of model often quickly simplifies as the data eliminates unsupported hypotheses.

We might note Pearl’s reference above to ‘cause and effect forces’ that are described by causal models. As noted in previous sections, system regulators generically overcome challenges to existence posed by physical law through the application of cause and effect forces. It is in this sense that the knowledge of inferential systems initiates a causal chain of forces which form and maintain systems within existence, and it is in this sense that inferential systems are autopoietic.
It is important to understand what Pearl means by causation:

For the purpose of constructing the diagram, the definition of ‘causation’ is simple, if a little metaphorical: a variable X is a cause of Y if Y ‘listens’ to X and determines its value in response to what it hears.

Although Pearl’s description is both clear and accurate, the term ‘listens to’ may be a little anthropomorphic when applied to abstract variables. Perhaps a better term is ‘receives information from’ and then his definition of causation becomes:

A variable X is a cause of Y if Y receives information from X and determines its value in response to that information.

In these terms, causation, as defined by Pearl, is an exact analog to inferential systems; models within inferential systems are updated by information or evidence in the sense that the value of the probability of each hypothesis composing the model is updated in response to the information or evidence it receives. This update or response has been generally described in terms of force; the evidence may be said to ‘force’ some of a model’s hypotheses to be assigned greater probability and some less (21). Thus, the model is shaped by the force of the evidence and we may say that the evidence causes the resulting model.

This analogy between causation and force is widely accepted. As Wikipedia tells us (132):

Causal relationships may be understood as a transfer of force. If A causes B, then A must transmit a force (or causal power) to B which results in the effect. 

In these terms, considering the close analogy between causation, inference and force, inferential systems may be considered the fundament form of causation. It is forces which cause effects and forces may always be understood in terms of the updating of inferential systems. This generic concept of force has been explored by Steven Frank (born 1957) who concludes that the process of inference may be considered as the force of data applied to models (21)

As we have seen in our previous discussion of physical forces, at the fundamental physical level a force is produced when information contained in a gauge boson updates the wave function of another quantum system. In Pearl’s terms we might say that the quantum system listens to the gauge boson and determines or updates its value, for example the value of its momentum, in response to what it hears.

Although nature has employed inferential systems since the beginning as its primary engine of existence, scientific understanding is only now becoming able to more fully describe them. The causal revolution provides our scientific understanding with important tools for describing the autopoietic aspect of inferential systems. In short, this new tool allows us to scientifically describe inferential systems as initiating a cascade of causes whose effects are existing systems. The mechanisms by which causes lead to effects may be understood in terms of orchestrated or regulated forces and these forces, as we have seen, are essential to overcome the many natural obstacles to existence.  The initiated causes must be highly orchestrated to achieve the effect of existence and knowledge required to achieve this orchestration is accumulated by the inferential system through the process of Darwinian evolution.  

A breakthrough made by the causal revolution is its understanding that a full scientific description of a causal process must include a causal model of the hypothesised causal cascade. Central to our understanding of inferential systems is that this same causal model is necessary to orchestrate or regulate existing systems as required by the good regulator theorem. In other words, the causal revolution has discovered that scientific descriptions of existing systems must include a causal model, and this is necessary because causal models are an essential component of existing systems; a scientific description of them that did not include a causal model would be incomplete. 

Central to the causal revolution’s understanding is that causation often involves interventions made to the normal or spontaneous unfolding of events. If an effect is caused, that means that the normal range of possible outcomes is constrained to a single outcome and this selection is described as occurring through an intervention in the normal course of events (128). In terms of inference, circumstances are causally orchestrated so that the received data updates the model to strongly predict or initiate a single outcome or effect.

At the core of autopoietic inferential systems is their ability to cause outcomes, to intervene in the spontaneous course of events and select a single outcome that might otherwise be extremely unlikely. As a biological example, we might consider that the knowledge contained in an organism’s genome causes specific outcomes or effects. A given three letter genetic codon identifies a single specific amino acid to be added to a protein and a gene, composed of a string of codons, identifies a specific complete protein. In turn, that protein, if it is an enzyme, may catalyze a specific bio-chemical reaction, an outcome or effect that is extremely unlikely to occur without the participation of the enzyme. This biological cascade of knowledgeable causes, resulting in specific effects, illustrates the tight causal control or regulation exercised by autopoietic inferential systems.

Pearl describes the causal revolution in terms of a ladder having three rungs which together fully describe causal systems. The first rung is characterized by correlations, the traditional study of statistics. The second rung is characterized by causal interventions of the type we have just examined. The third rung involves counterfactuals or hypothesis regarding what might be rather than what is. A dictionary example of a counterfactual hypothesis is ‘If kangaroos had no tails, they would topple over (133).

While causal interventions describe the autopoietic aspect of inferential systems in that they are required to ensure that the system’s knowledge causes a specific process of self-creation and self maintenance, counterfactual hypotheses describe the evolutionary aspects of inferential systems as they make hypotheses which have never been tested before. Counterfactual hypotheses are tools which may be used to search through the space of possibilities, a search for those possibilities which may be made actual, which may be brought into existence. 

For example, the counterfactual hypothesis involving kangaroos and their tails may be coded in the genome of a kangaroo where a mutational cause produces the effect of a tailless kangaroo offspring. The mutant gene codes the counterfactual hypothesis which in effect asks if a kangaroo with no tail would be reproductively successful (didn’t always fall over). If the resulting tailless kangaroo did achieve reproductive success the possibility would be made actual and tail-less kangaroos would achieve existence in the world.

The causal revolution has extended scientific understanding of causation beyond correlations to include causal models, interventions and counterfactual hypotheses. I have provided examples from biology to illustrate how this extended understanding describes actual causal processes central to biology and in chapter 3 we will see that this understanding is also central to domains of reality other than biology. In effect autopoietic inferential systems in all domains cause their own existence and evolve through their exploration of counterfactual hypotheses.

However, it appears that those who forged the causal revolution consider it more of a revolution in scientific calculation than in the understanding of actual phenomena found in nature. Pearl, for example writes extensively on how causal models can assist the computation of relationships between variables but nowhere does he suggest that these models actually exist in the natural world or their essential role in bringing actual phenomena into existence (129). Instead he understands the findings of the causal revolution as calculational devices that allow us to calculate data and arrive at causal conclusions.

This is a common misunderstanding when a scientific revolution first brings some important new physical phenomena to light for which there is little direct observational evidence. It occurred, for example, in the late 1800s when many considered the controversial concept of atoms as referring to a shorthand for making scientific calculations rather than to actual phenomena. When Boltzmann tried to publish his work deriving thermodynamics from atomic theory, his journal editors insisted that he refer to atoms as ‘bilder’; merely models or pictures that were not ‘real’ (134). Again, in the early 1900s when Mendel’s concept of genes was rediscovered there ensued a great debate among biologist on whether Mendel’s theory conflicted with Darwin’s. The one thing agreed upon by most leading biologists was that Mendel’s ‘genetics’ was not a physical process but merely a means of making calculations. As recounted by the philosopher of science David Hull (37):

As much as Bateson might disagree with Pearson and Weldon about the value of Mendelian genetics, he agreed with them that it was unscientific to postulate the existence of genes as material bodies. They were merely calculation devices.

As in the case of atoms and genetics I am confident that history will demonstrate that the new calculational methods of the causal revolution describe actual physical reality. The task of science is to describe nature and when new calculational tricks are discovered that provide powerful methods of describing reality at a deeper level, we may rest assured this is because nature also uses those same tricks to create and maintain the actual phenomena, in other words, that science has merely rediscovered and described nature’s own methods. This is the proper role of science.

References

1. Causal inference in statistics: an overview. Pearl, Judea. s.l. : Statistics Surveys, 2009, Vols. Volume 3 (2009), 96-146.

2. Pearl, Judea and Mackenzie, Dana. The Book of Why: The New Science of Cause and Effect. s.l. : Basic Books, 2018. ISBN-10: 046509760X.

3. Jaynes, Edwin T. Probability Theory: The Logic of Science. s.l. : University of Cambridge Press, 2003.

4. Harari, Yuval Noah. Sapiens: A brief history of humankind. s.l. : Harvill Secker, 2014.

5. Boyer, Pascal. Minds make societies: How Cognition Explains the World Humans Create. s.l. : Yale University Press, 2018.

6. Universal Darwinism as a process of Bayesian inference. Campbell, John O. s.l. : Front. Syst. Neurosci., 2016, System Neuroscience. doi: 10.3389/fnsys.2016.00049.

7. Dennett, Daniel C. Darwin's Dangerous Idea. New York : Touchstone Publishing, 1995.

8. Blackmore, Susan. The Meme Machine. Oxford, UK : Oxford University Press, 1999.

9. Bayesian Methods and Universal Darwinism. Campbell, John O. s.l. : AIP Conference Proceedings, 2009. BAYESIAN INFERENCE AND MAXIMUM ENTROPY METHODS IN SCIENCE AND ENGINEERING: The 29th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering. AIP Conference Proceedings, Volume 1193. pp. 40-47.

10. Simple unity among the fundamental equations of science. Frank, Steven A. s.l. : arXiv preprint, 2019.

11. Wikipedia. Causal reasoning. Wikipedia. [Online] [Cited: May 26, 2019.] https://en.wikipedia.org/wiki/Causal_reasoning.

12. Google dictionary. Counterfactual. Google dictionary. [Online] [Cited: June 2, 2019.] https://www.google.com/search?q=counterfactual&oq=counterfactual&aqs=chrome..69i57j69i59l2j0l3.8063j0j8&sourceid=chrome&ie=UTF-8.

13. Wikipedia. Ludwig Boltzman. Wikipedia. [Online] [Cited: June 7, 2014.] http://en.wikipedia.org/wiki/Ludwig_Boltzmann.

14. Hull, David L. Science as a Process: An Evolutionary Account of the Social and Conceptual Development of Science. Chicago and London : The University of Chicago Press, 1988.



[1] If in doubt as to the widespread use of this meme, try googling it.