Causal Graphs and Concept-Mapping Assumptions

Abstract

Determining what constitutes a causal relationship between two or more concepts, and how to infer causation, are fundamental concepts in statistics and all the sciences. Causation becomes especially difficult in the social sciences where there is a myriad of different factors that are not always easily observed or measured that directly or indirectly influence the dynamic relationships between independent variables and dependent variables. This paper proposes a procedure for helping researchers explicitly understand what their underlying assumptions are, what kind of data and methodology are needed to understand a given relationship, and how to develop explicit assumptions with clear alternatives, such that researchers can then apply a process of probabilistic elimination. The procedure borrows from Pearl’s concept of “causal diagrams” and concept mapping to create a repeatable, step-by-step process for systematically researching complex relationships and, more generally, complex systems. The significance of this methodology is that it can help researchers determine what is more probably accurate and what is less probably accurate in a comprehensive fashion for complex phenomena. This can help resolve many of our current and future political and policy debates by eliminating that which has no evidence in support of it, and that which has evidence against it, from the pool of what can be permitted in research and debates. By defining and streamlining a process for inferring truth in a way that is graspable by human cognition, we can begin to have more productive and effective discussions around political and policy questions.

1. Introduction

Causal inference is a key goal for understanding the relationships among phenomena in the real world that researchers are attempting to study [1] This becomes a challenging task when possible causal phenomena are numerous, highly interrelated, complex, and complicated to study with validity [2,3]. As things currently stand, there is no clear method for either promoting correct facts and high quality and honestly treated evidence, or for eliminating incorrect facts and inferences of poor quality, or dishonestly treated evidence from the pool of knowledge that is acceptable in policy debates. This paper proposes a possible method to clarify researchers’ intentions and work, determine what data are necessary to collect, guide the selection of the methodology of treating the evidence, and produce possible counterfactual arguments that can be tested to establish a greater probability that correct inferences are drawn from the data. The hope of this paper is to clarify what is more probably true from what is less probably true and to streamline the pool of evidence that is permissible in policy and political debates. High quality and honestly treated evidence gains precedence over, and is promoted in discussions and debates, at the expense of poor quality and dishonestly treated evidence.

2. Literature Review

“Causality” is defined as “the relationship between something that happens or exists and the thing that causes it” [4]. Determining causal relations among variables is a challenging and much studied topic [1,5,6,7,8,9,10]. Much of the literature on causal relations comes from the medical field of epidemiology [11] and is used to infer causal relationships in disease diagnoses and treatment effects of medical regimens [12]. Causality is also a much studied subject in the social sciences. Its inference is typically derived from a statistical method or technique or qualitative analysis [6,13,14,15,16,17]. Testing for Granger Causality, which is a statistical concept where variables that cause effect variables contain information that predicts effect variables within them, has used “path diagrams” in the literature [18]. This paper specifically draws upon the concept of the “causal graph” described by Pearl [19] as the basis of this methodology. The causal graph is used alongside “concept mapping” in order to tease out the underlying assumptions about the nature and relationships among the variables in question. Casual graphing and concept mapping promote better understandings of the researchers’ assumptions, and they develop alternative counterfactual cases with different causal graphs. Causal graphing also could help design research to test the factual and causal validity of the causal graph and, by extension, the researchers’ concept map [20]. In summary, the researchers and other stakeholders may make different concept maps and causal graphs according to existing methodologies. The difference with this proposed method is that it actively seeks to remove all or parts of concept maps and causal graphs to infer what is more probably true in the real world itself.

A “causal graph is a directed graph that describes the variable dependencies” [21]. Causal graphs were first developed in the fields of mathematics, computer science, machine learning, and statistics [1,22,23] but have since evolved to the study of complex phenomena, such as epidemiology [9] and planning [21,24,25]. While causal graphs are not new tools in several academic fields and have been used in statistical analyses for developing causal relations after the data collection, it does not appear that they have been widely used by researchers to sketch assumptions and hypotheses before the data has been collected.

The ideas expressed here are not new in the field of economics. One of the first two Nobel Laureates in Economics in 1969, Jan Tinbergen, collected all proposed macroeconomic models in the late 1930s and built models of the business cycle with a similar technique ([26] pp. 101–130). Tinbergen “explained his model building as an iterative process involving both hypotheses and statistical estimation” ([26], p. 103). Morgan (1990) points out that “Despite their usefulness, few copied his graphical methods” ([26], footnote 9, p. 111).

While Tinbergen’s methods are similar to the concept of causal graph modeling that is described here, they are not quite the same. Tinbergen was aiming to understand economies and processes in economies, not to infer causal relations among different social, economic, ecological variables, and factual conditions. Indeed, the method that is described in this paper is more applicable in meta-analyses of existing studies and guiding the direction of future research, not as the centerpiece of individual topical studies. The intention behind this method is to understand what is true and what is not as true, and to provide a quantitative method for deriving those truths and assessing the quality of the evidence behind them.

Another process that is similar to this one is known as “group model building” [27,28,29]. Group model building is a process that was created by system dynamics researchers to facilitate diverse stakeholders sharing information across different fields. This is done to solve problems that are common to these stakeholders by unifying, standardizing, and connecting the information that is presented by and for the stakeholders in question [27,29]. While this is a useful technique for helping groups understand problems from many different angles, it is not a generalized way of inferring causality and truth. Creating and testing different causal graphs with the evidence that is available is a separate process that aims to produce general knowledge of empirically inferred reality. The goal with causal graph analyses is to produce a coherent and accurate map of a given concept or problem that is more probably true than competing alternative maps. It is the process of weeding out models that are not supported by evidence, more than it is just the production of different models.

Most people have implicit assumptions about how the world works, in addition to possible desires about how they would like the world to work [15,30,31,32]. One method for determining the underlying assumptions that are implicit in a research project is to map them out through a process known as “cognitive mapping” [33,34,35,36,37]. Cognitive mapping has been used to understand the implicit assumptions and decisions made by policymakers in the past [38]. Cognitive maps give rise to different concept maps, which then are used to produce different causal graphs. It is logically plausible that the creation of causal graphs in causal modeling is produced by the conceptual maps people make by the same cognitive maps used by researchers, policymakers, and stakeholders. Indeed, cognitive mapping is implicit in some research concerning Bayesian networks, which map out the probabilities that a set of causal conditions relates to a set of observed variables [39,40,41,42,43]. It also has been linked to modeling ecological systems by researchers [37].

The hypothesis that underpins this perspective is that implicit cognitive maps of researchers, policymakers, and stakeholders alike result in the production of different conceptual maps of the world. The interplay between cognitive maps and conceptual maps gives rise to different causal graphs being produced through the different perceived factual “nodes” (points) and connecting “edges” (relations or connections between factual nodes) of the researcher, policymaker, and stakeholder. This is different from existing methods for making the goals of researchers, policymakers, and stakeholders explicit, such as the Logic Model. Logic models display the connection between different inputs and activities with different outputs and outcomes [44], in that this method is more free-form and allows the cognitive maps and implicit biases to be made more readily apparent instead of confining the maps to a preset form. Different assumptions, perspectives, levels, and degrees of awareness in the cognition of researchers, policymakers, and stakeholders result in the perception of different “facts”, different interpretations of those “facts”, and different edges among the “facts”. This could be done implicitly and subconsciously by the researcher and policymaker, but it also is hypothetically possible for it to be done deliberately through conscious choice and selection of facts, interpretations of facts, and edges among facts [10,32,45,46].

A hypothetical example of this is between people who identify as “conservatives” and “liberals” looking at the same situation facing their shared nation. The conservative may claim that the moral integrity of the society is eroding as time passes, while a progressive may have a different outlook on change and difference in a society from one time to another. The evidence suggests that people on both sides will look for, perceive, and interpret the situation differently in mutually exclusive way. For example, the environment cannot both be and not be affected by humans’ economic activities, and it cannot both be and not be significant for human survival. Different problems are identified, different choices and preferences are made, and different actions are seen as more or less acceptable because of those differences between the general psychological phenotypes. The obvious problem with relying on these subconscious assumptions and biases alone is that the individual person who is making the policy decisions may not accurately understand, represent, or interpret the meaning of the world. Without an accurate map of how the world works, policymakers are less able to make the best possible choices for the people living in the society and for their own benefit as policymakers making decisions that affect the world they live in. One can think back to the times before navigational and weather/oceanographic sensory technology had advanced to the point where ships could orient themselves accurately on Earth. Without the production of these technologies, which aid navigation and the ability to detect and predict conditions around the ships, sailors’ lives were easily lost on the tempestuous oceans, and valuable cargo was lost and destroyed in transit around the world more often than now. The analogy could be carried over to the fate and condition of nations and human societies.

3. The Methodology

The goal of this paper is not to advocate a singular methodology or tool for studying complex phenomena in our universe. Rather, the goal is to propose a new tool that can be used to help determine the appropriate tool(s) for studying complex phenomena, and to at least partially overcome the deficits of human cognition and perception in research and decision-making. By making the assumptions explicit rather than implicit in research and policy designs, we can get a firmer grip on what healthy priorities are, and how to achieve them. Below is an example of a theoretical causal graph (Figure 1).

where x factors cause y effect when brought together in this combination. We see that x1 causes x2 which, when combined with x3, produces y1 effect. The plus represents x2 having a positive feedback effect on y effects (more of x2 leads to more of y1), while x3 has an unknown effect on y1. Notice the error/unknown factors variable to account for anything else the model misses.

The methodology is simple to describe and works as follows:

• Draw out the causal graph as the researcher perceives it to be. This is the conceptual mapping stage, since all causal graphs are ultimately concept maps. Nodes or points in the graph are facts or conditions, edges between the nodes are interactions and associations among the facts. The researcher should be free to base this initial step on their own working knowledge, the existing literature on the subject in question, and any applicable theory;
• Consult with other researchers, policymakers, and stakeholders to develop alternative facts and conditions and alternative ways for them to interact with each other through the interaction edges in the graph;
• Design research projects to test the validity of the factual nodes and interaction edges that are produced from Steps 1 and 2:

  a

  It is important to note that this paper is agnostic about the specifics of the designs of the research, so long as it is logically valid and testable;

  b

  This is where any number of qualitative and quantitative methods can be used;

  c

  It is also a good idea to use multiple methods on the same factual node or interaction edge to increase the probability of validity. That is often called robustness in research;

• Out of the population of causal graphs that were created, assign equal probabilities that each one is valid based on the total number of causal graphs that are explored.

  a

  The probability of the population of causal graphs can never truly equal 1 for complete validity because there is always an unknown quantity of potential error present in the population of models, i.e., the unknown unknowns;

  b

  The probabilities can be explicitly Bayesian, empirical Bayesian, or based on flat priors;

• Consider the quality and source of the evidence that is presented. If quality evidence for a particular edge or node is present, then that adds to the probability that that edge or node is true at the expense of other edges and nodes. If there is evidence against a node or edge, it subtracts from the probability that that edge or node is true without necessarily affecting alternative edges and nodes. Poorer quality evidence has less of an effect, or no effect on the probability of demonstrating truth;
• Alter the probability of validity for each of the graphs as evidence becomes apparent through new research. This can be based on Bayesian updating or frequentist testing;
• Repeat Steps 1 through 6 using a variety of techniques to examine each node and each edge in the causal graph.

It is important to again note that this procedure is agnostic about the specific research techniques that are used to infer causality or the truthfulness of factual nodes. Notice how the factual validity for each of the variables (the nodes in the causal graph) and causal edges (the links in the causal graph) are not necessarily known, and are rather hypothesized to exist based on past evidence and the circumspection of the researcher. From this model, we can derive various other models to test for and identify possible methods for gathering and examining the data. We can see other possible models below (Figure 2 and Figure 3).

Notice how parts of the graphs in Figure 2 and Figure 3 changed from Figure 1, representing different and mutually exclusive hypothetical models that may or may not be more accurate than the original hypothesis.

These assumptions (that are different from the original causal graph) each then have their own theoretical and observational bases and their own interpretations of what is present and happening in the real world outside of the researcher’s perspective and assumptions. With this technique, it is also possible to model unknown or hypothesized interactions and facts, such as the question mark between variables x₃ and y₁. Other models can be constructed using all of the possibilities. For simplicity’s sake, most of these options in the research design space have been left out. However, if the researcher(s) are able to get the largest possible collection of causal graphs together while staying relevant to the topic(s) at hand, the larger design space should provide a rich environment for testing the factual assumptions and interactions among the variables. Researchers can then work together across disciplines to design experiments and determine which data to collect and how in order to “shave away” at the hypothesis space of the research topic. The surviving causal graphs, which withstand the scrutiny of the researchers’ efforts, can be said to be more probably true and valid than the other causal graphs that have aspects that are not valid or which have little to no evidence in support of them. These surviving causal graphs correspond to Bayesian posteriors or unrejected frequentist hypotheses, in that they are the end products of analyses.

Figure 4 is an example of a causal graph produced to clarify questions about education policy and the factors that link in to create academic, social, behavioral, and personal success in students. Using various data sets and methods of estimation, the most likely causal pathways could be found. Some researchers will add double-headed causal arrows and reversed arrows.

There are two ideas that can be deconstructed from taking this holistic approach to education and educational success. The interrelated subject areas, such as the defined pedagogy, territorial demographics, the political environment, and parental/familial conditions that the child grows up in can be broken out from the causal graph into their own interrelated clusters as part of the larger graph that contains the whole. This would enable collaboration among experts in these various fields to create a more accurate model of the whole picture of how children develop, learn, and grow into adults, which can then give us a more accurate map for helping policymakers be better able to see where and how they might intervene in the given subject area. The second idea is that the whole causal graph is malleable to the perspective of the researcher in question, and alternatives for hypothesis testing can easily be developed by simply going through the graphical representation of the problem(s) at stake to find other possibilities and alternatives. Time stamps can be added to refine the temporal relationships among the variables.

4. Implications of this Method for Policy Research

The implications of this method for conducting social science research would allow policy researchers, policymakers, and stakeholders to better understand not only their own implicit, subconscious biases and explicit conscious biases, but to move beyond those biases in order to perceive and study our complex social and environmental worlds accurately. Communication of divergent beliefs and models would be easier. It is feasible that policymakers, and the researchers and stakeholders as well, will be able to move beyond disagreements over what may be just different cognitive maps in order that better choices may be made faster, with less debate, and with greater efficacy than would ordinarily happen without using this methodology explicitly to understand, design, and analyze situations and conditions in our social and environmental worlds. At least, it would be clearer what issues must be resolved and models estimated.

The methodology can also be used as a technique for holding policymakers and researchers more accountable for their assumptions and their chosen research techniques. Even though the method itself is agnostic about the methods that are used in research, there are practices for testing validity and causality that are more or less effective than others. By explicitly drawing the causal graph, it is easier to tell whether more or less appropriate methods are being used to test the nodes and causal edges of the graph. By explicitly stating the implicit and explicit biases of the individual through the process of mapping out their factual and causal assumptions, human societies and organizations that adopt this method for making choices and understanding the world may be able to more effectively understand political opponents’ concerns and perspectives, as well as to be more effectively able to challenge those perspectives and opinions that are not based in evidence both behind closed doors in negotiations and in front of the fora of the general public. Assumptions totally lacking empirical verification would stand out.

The most significant benefit of using this methodology is that mutually exclusive opinions on facts and relations can be more clearly examined. Most of the common controversies in policy debates stem from competing, mutually exclusive ideas on how the world works, and how it ought to work for human well-being and survival. From whether to have public sector involvement in the economy, to the necessity of protecting the environment, the different attitudes, biases, and opinions cannot all be called of equal value for ensuring human health and well-being. Causal graphs can be used to sort through those differences in policy preferences and opinions to deliver a clearer picture of common reality and what is needed for human societies at given times. Those opinions that are supported by quality evidence can then take priority over those that simply are not, or have evidence against their empirical validity.

5. Caveats to this Method

The most glaring problem with this methodology is that it does not give instructions on how to collect data, what data to collect, or how to treat the data when they are collected. It may help inform research decisions, but it does not give explicit instructions on what to use or when to use it. This leaves the design of the experiments still open to possible researcher bias and the usual difficulties with inferring causality with researchers who have underlying assumptions and cognitive biases that they consciously or subconsciously prefer over other models and methods. Through explicitly stating the researcher’s hypothesis space and cognitive bias, measures of robustness can be developed for causal models to see if researchers are truly ruling out other possibilities or whether they are honestly adhering to sound da identification, collection, and interpreting methods. Ignoring logical possibilities would be much more difficult.

Another caveat to this research methodology is the possibility for aspects of the causal graphs to change stochastically during the development of the models and throughout the experiments and analysis of the data. That is, the structure can change. A policymaker may be in the middle of developing a causal graph which is presently accurate, but may have dynamic aspects to it which can change in the near to distant future as aspects of our social world (such as technology and our understanding of the world itself) change. In addition to these probable knowledge based changes, there may also be some aspects of our social world which change due to aesthetic preferences or changes in relative perspective and attitude. This further complicates the development of these causal graphs, as the aspects and perspectives of them may change in ways that do not track neatly with the development and production of our knowledge and awareness. What may be in fashion and perceived of as desirable now may not be viewed as such in the near to distant future, thus altering the perspectives on the causal graphs that are developed today or rendering them potentially useless for achieving the goals of the society in the future. Thus, the dynamic and evolving nature of consciousness and preferences over time may influence the development of these causal graphs, if not affect the actual graphs themselves in the content of their facts, interpretations of facts, and interactions among the variables. In response, more basic social factors related to group dynamics can be added to the models, such as fundamental psychological processing common to humans. Change itself can become a part of the model. As different edges and nodes can change over time, and their changing nature can theoretically be observed, the changes and their effects can be noted and tested. This gives the resulting models significantly greater empirical validity, and thus enriches our understanding of common reality to the fullest possible extent that we can achieve.

A third possible problem with this methodology is that there is no method for keeping the model parsimonious and simple. While this may not be a problem when working with large and complex topics, it can be said that it is feasible that the models that researchers may make could become too unwieldy for practical use. A simple method for resolving this while not abandoning the potential complexity in a subject is for the researcher to narrow their focus initially to a given factual node or a specific interaction, and then to grow the model from there, limiting it to the practical relevance of the research in question. The researcher in question, or other researchers can then expand the web of knowledge in other directions at future times.

6. Conclusions

This paper presents a new tool for researchers and policymakers alike for understanding complex and interconnected topics of interest and importance to human society as a whole. Through explicitly stating the assumptions behind the subject, researchers and policymakers can then develop counterfactual alternative graphs for the subjects of their interest and research, identify data that is relevant to the subject, develop methods for collecting and analyzing the data, and then systematically shave away at factual assumptions and hypothetical interactions for which there is little to no quality evidence. Through this deductive process of elimination, researchers and policymakers alike can eliminate graphs for which all or parts do not have evidence, and thus, be left with a pool of possibilities that shrinks in size and increases in the chances of being probably accurate representations of reality itself.

It is possible that some specific aspects of the graphs may change over time with peoples’ attitudes, preferences, and perspectives. However, it is assumed explicitly in this paper that the underlying method of creating causal graphs with fact nodes and interaction edges can be valid throughout time, space, and context, even if the specific models change over time. The process of shaving away at conceptual maps with this method can produce a more robust, accurate, and complete representation of reality that the human mind can comprehend and use for other purposes. By doing so, we can begin to constructively resolve key policy and political debates as they arise with this common process of gathering, analyzing, and evaluating evidence from our common reality. The political debates may be based ultimately in values and opinions. However, not all opinions and values are of equal value for human society’s health and well-being. This proposed method hopes to help resolve these debates for that which is factually true and healthful, at the expense of those opinions that are not true, and are very likely unhealthful for humans in general.