Causal Inference: Models and Methods

Background and significance. The development and testing of innovative solutions to social problems require that we understand causal relationships between manipulable social policies or social work practices and outcomes. Credible inferences of causality require that social work science be conducted using methods that are appropriate for the data and assumptions about how the data were generated.

Causal frameworks. Lord’s Paradox, in which two statisticians employing different assumptions obtain different answers to the same causal question using the same data, is a classical problem that illustrated the importance of credible assumptions to causal inference. Several different causal frameworks help to clarify such assumptions. Campbell’s model of validity, with its inventory of threats to internal validity, is well known in social work research. Another well-known framework is the Rubin potential outcomes model, which uses a compact statistical framework to represent causal assumptions. In both models, unobservable variables and their role in confounding the effects of treatment on the outcome are central. A third model, Pearl’s directed acyclic graph (DAG) is a complex graphical approach. DAGs have pedagogical utility, clarifying that to estimate an unbiased treatment effect, the relationship between a confounder and either the treatment or the outcome must be controlled for; it is not necessary to do both.

Methods. Randomized control trials are the gold standard, providing a benchmark for all non-randomized methods for causal inference such that when applied appropriately, these non-random methods render treatment effects “conditionally random”. These non-random methods span a spectrum of familiar and novel approaches. On one end of this spectrum, regression and structural equation models are general approaches that require the strong assumption that the confounders of treatment are measured. On the other end of the spectrum, regression discontinuity directly captures the non-random assignment mechanism, enabling unbiased treatment effect estimation.

Between these extremes lie a number of approaches each having strengths and weaknesses. In propensity score matching the probability of treatment in both arms is estimated and used to balance the groups on observable characteristics. Instrumental variable estimation is a method in which a variable not associated with confounders but highly predictive of the treatment must be identified. Fixed effects models are methods typically used by economists in panel or clustered settings in which all invariant characteristics of a person or context are controlled for. Finally, in interrupted time series analysis, a transition point coinciding with implementation of a treatment is modeled on long-term trend data. Many of these models are related; for example, certain types of regression discontinuity designs are also instrumental variable models.

Implications. To be used to inform innovative social work practices and social policies, inferences from causal analysis must be seen as credible by the target population: the practitioners and policymakers with authority to implement these innovations. Researchers must be able to explain the assumptions needed to draw causal inferences from their studies given the design and data, and must choose methods that are appropriate for these conditions and provide the most solid evidence of the efficacy of treatment.