Session: Applying Propensity Score Analysis to Challenging Research Questions Concerning Causality (Society for Social Work and Research 15th Annual Conference: Emerging Horizons for Social Work Research)

Randomized clinical trials (RCTs) have long been considered the gold standard for program evaluation and causal inference. However, in many practice settings, particularly in social work evaluation and research, RCTs are infeasible and unethical. In the past 30 years, researchers have recognized the need to develop more efficient approaches for assessing treatment effects from studies based on observational data, i.e., data derived from studies in which random assignment is not used or is compromised. This growing interest in seeking consistent and efficient estimators of causal effects led to a surge in work focused on estimating average treatment effects under various sets of assumptions (e.g., Heckman, 1978, 1979; Rosenbaum and Rubin, 1983).

As a part of this effort, researchers have found that the conventional covariance control approach has numerous flaws and, in many circumstances, should be replaced by more rigorous methods to draw causal inference. For instance, Sobel (1996) criticized the use of dummy variables (i.e., treatment versus nontreatment) to evaluate treatment effects in regression models (or regression-type models). The primary problems with the covariance control approach are: (1) the dummy treatment variable is specified by these models as exogenous, but in fact it is not, and determinants of incidental truncation or sample selection must be explicitly modeled when estimating causal impacts on outcomes (Heckman, 1978, 1979); (2) the strongly ignorable treatment assignment assumption (i.e., conditional upon covariates, treatment assignment is independent from outcomes under both treatment and control conditions) is prone to violation in observational studies; under such a condition, endogeneity leads to biased and inconsistent estimation of regression coefficients (Berk, 2004; Imbens, 2004; Rosenbaum & Rubin, 1983); and (3) covariance control does not automatically correct for nonignorable treatment assignment (Guo & Fraser, 2010). Given the wide spread use of covariance control, advances in methods of causal inference have profound implications for social work research.

The purpose of this symposium is to illustrate the utility of propensity score analysis (PSA) for strengthening causal inference in intervention research. Presenters will use three distinct projects to show how internal validity threats may be limited and controlled using PSA. The substantive areas of these studies cover a broad range of topics in social work research, including the prevention of aggressive behavior in childhood, the impact of marriage on transition into low-income homeownership, and of the treatment of male sexual offenders in prison. A broad range of PSA models are employed by these studies, including nearest neighbor matching, optimal matching, estimation of propensity scores using generalized boosted regression, matching estimators, PSA in conjunction with survival analysis, and Rosenbaum's sensitivity analysis to discern hidden selection bias. Each study will focus on challenges and strategies in drawing causal inference, as well as the limitations of PSA that require discussion when presenting findings.