Contents: The workshop begins with a discussion of the sources, types, and consequences of selection effects on causal inference. Hidden selection is virtually not a problem in classical randomized clinical trials, because the mechanism of randomization balances data for both measured and unmeasured variables to make the average of omitted variables equal to zero. Researchers using nonexperimental data should make extra effort to deal with hidden selection bias.
It then describes the Rosenbaum's approach of sensitivity analysis. The core idea of such approach is to use existing data to estimate the level of bias. Although selections are hidden and in a black box, researchers may estimate the sensitivity of study findings to hidden selections, and gain sufficient information for improving the study in future. Using a well-designed spreadsheet, the workshop illustrates the basic idea for one of the Rosenbaum's models (i.e., the Wilcoxon's signed-rank test for sensitivity analysis of a matched pair study). It also demonstrates the implementation of such analysis using Stata rbounds program.
It then illustrates the application of the sensitivity analysis to a social work study evaluating the impact of poverty on children's academic achievement. Using the 1997 national survey data of Child Development Supplement, the study conducted an optimal matching using six covariates (i.e., caregiver's education, caregiver's use of AFDC in childhood, and demographic variables), and found that in 1997, children who used AFDC had an average letter-word identification score that was 3.17 points lower than that of children who never used AFDC (p<.05). Using Rosenbaum's approach, the sensitivity analysis shows that the study is sensitive to hidden bias at a sensitivity level of 1.43. Because 1.43 is a small value, the analysis reveals that the study finding is very sensitive to hidden bias, and therefore, further analysis that controls for additional biases is warranted.
The workshop concludes by summarizing potential areas in social work research that should routinely consider using the sensitivity analysis to discern the level of hidden selection bias.
Pedagogical Techniques: Teaching methods include lecture, PowerPoint presentation, and Excel/Stata demonstration.