Optimal Propensity Score Matching and Its Applications to Social Work Evaluations and Research

Purpose: Social work researchers often face challenges in estimating causality. Rosenbaum and Rubin (1983) developed an innovative method “propensity score matching” (PSM) to estimate treatment effects that controlled for selection bias. Although the approach is robust and efficient, its limitations have not been addressed until recently. A common limitation in PSM is its requirement for a reasonably sizeable common-support region to perform nearest neighbor matching within caliper (also known as greedy matching). In real applications, if data do not meet this requirement, PSM is not feasible. Rosenbaum (2002) and others have developed a new approach called optimal propensity score matching (OPSM) to address the problem. An R program called optmatch was developed to run OPSM (Hansen, 2007). This workshop employs the 1997 Child Developmental Supplement (CDS) data as an example to demonstrate the application of OPSM to a study addressing the impact of poverty on children's academic achievement. A key feature of the illustrating study is its removal of more selection bias in a more flexible fashion than conventional approaches.

Contents: The workshop will focus on the following topics: (1) review of the OPSM procedure, (2) the problem of greedy matching, (3) the network flow theory and its application to OPSM, (4) three types of OPSM – full matching, variable matching, and pair matching, and (5) post-matching analysis.

Findings: Using the 1997 CDS data, the illustrating example shows that the estimates of poverty effects produced by conventional methods are biased and exaggerated. A t-test shows that the mean difference of letter-word-identification score between children who ever used AFDC and children who never used is -9.82 points (p<.0001). An OLS regression estimates the difference as -4.73 points (p<.0001). However, an optimal full matching using the Hodges-Lehmann aligned rank test indicates that such difference is only -3.00 points (p<.01), and an optimal pair matching followed by a difference-score regression shows the difference as -3.17 points (p<.05). With the given data, we even cannot perform a nearest-neighbor matching because the common-support region is too narrow.

Implications: This workshop is a step-by-step demonstration of the methods depicted by the seminal paper of Haviland, Nagin, and Rosenbaum (2007), which underscored the importance of combining propensity score matching and group-based trajectory for observational studies. Our study supports their conclusion. To evaluate causality or treatment effectiveness using observational data, researchers simply cannot afford to neglect biases produced by regression or any regression-typed models.

Pedagogical Techniques: Teaching methods include lecture, PowerPoint presentation, and computer demonstration of running R optmatch.