Content: The workshop is organized in three parts. This workshop opens with a general discussion regarding the role of observational data, theory, and causal analysis in the what-works context. The discussion also considers how RCTs and observational studies are linked in the knowledge-building enterprise. The second and third parts focus on the practical application of these ideas. For theoretical orientation, we will introduce the concept of the gatekeeper in social work practice, and the implications of unmeasured gatekeeper selection tendencies for causal inference in non-experimental evaluation studies. To that end, we will introduce and describe the estimated best linear unbiased predictor (EBLUP) as a method for estimating latent gatekeeper effects, which has potential for broad application across social work research. We will demonstrate how to compute the EBLUP from mixed model estimates of treatment assignment, and discuss its benefits as a measurement approach. To demonstrate the benefits of the proposed approach, we will provide an example of using the EBLUP in a child welfare evaluation scenario using simulated data. We will follow that with the results from a real-world evaluation that used a random effects model to generate an adjusted estimate of worker referral tendencies. The discussion will highlight how random effects were then used to assess the efficacy of an intervention.
Implications: Answers to the what-works-question fit within the broader context of social work knowledge. The task of building that knowledge will, in the long-run, rely on a mix of evidence derived from both observational as well as experimental studies. The strength of evidence built will depend on how theory is used to generate questions that are then linked to methods best suited to the question rather than the other way around. In this workshop, participants will learn how theory and observational data come together to improve our understanding of treatment effects. Finally, because the methods are broadly applicable, implications for causal modeling over a broad range of social work issues will be addressed.