Abstract: Using a Directed Acyclic Graph to Inform Research with Propensity Score Analysis (Society for Social Work and Research 27th Annual Conference - Social Work Science and Complex Problems: Battling Inequities + Building Solutions)

All in-person and virtual presentations are in Mountain Standard Time Zone (MST).

SSWR 2023 Poster Gallery: as a registered in-person and virtual attendee, you have access to the virtual Poster Gallery which includes only the posters that elected to present virtually. The rest of the posters are presented in-person in the Poster/Exhibit Hall located in Phoenix A/B, 3rd floor. The access to the Poster Gallery will be available via the virtual conference platform the week of January 9. You will receive an email with instructions how to access the virtual conference platform.

Using a Directed Acyclic Graph to Inform Research with Propensity Score Analysis

Friday, January 13, 2023
Desert Sky, 3rd Level (Sheraton Phoenix Downtown)
* noted as presenting author
Yali Deng, MSW, PhD student, University of Maryland at Baltimore, Baltimore, MD
Nikita Aggarwal, MSW, PhD student, Graduate Research Assistant, University of Maryland at Baltimore
Vashti Adams, MSW, Doctoral Student, University of Maryland at Baltimore, Baltimore, MD
Background and purpose: Social work researchers seeking to estimate the causal effect of non-randomly assigned treatment frequently utilize propensity score analysis (PSA) to balance the distributions of confounding variables among treated and untreated participants. Given that a key assumption in PSA is that measured variables account for the influence of unmeasured variables, researchers may naively include all possible confounders into the adjustment stage with little consideration of the associations between confounders, the treatment, and the outcome. This may induce bias, particularly if variables only associated with the treatment and not the outcome are used. Directed Acyclic Graphs (DAGs) provide a visual representation of the causal relationships between variables as informed by theory and existing literature, can be used to ensure PSA is the best method in a given context, and can guide selection of predictors in both stages of PSA. In this presentation, we present an example of a PSA study in which a DAG was essential to making these decisions. This is an application of the third step of using a DAG to inform social work research design and analysis.

Methods: This exemplar study used a quasi-experimental study design to estimate the causal effect of student gatekeeper training (i.e., treatment) on their use of counseling services for mental health problems (i.e., outcome). A DAG was built to visualize the association between gatekeeper training, counseling utilization and a set of confounders according to theory and empirical studies. This visualized tool informed PSA in three ways: (1) no other method (e.g., instrumental variables) seemed adequate for estimating an unbiased effect; (2) the selection of predictors of the propensity score model was informed by a DAG which shows an extensive list of correlates of counseling utilization (e.g., gender or mental illness diagnosis); (3) the DAG was used to identify the minimal sets of covariates that needed to be included in the outcome model. Secondary data was retrieved from the Healthy Mind study 2020-2021, a cross-sectional study investigating mental health status, service utilization, and related factors among university students.

Results: We used the propensity score model, guided by the DAG, to predict the receipt of gatekeeper training, which balanced the treatment group and comparison group across a set of confounders. Finally, to answer our research question, a logistic regression model adjusting for covariates was conducted by using propensity score weighting. By using PSA, we found that receiving student gatekeeper training was associated with 19% higher odds of counseling utilization compared to students who did not receive gatekeeper training.

Conclusion and implications: PSA is widely used throughout social work research to estimate unbiased causal estimates. The risk of inducing bias is ever present, however. The DAG framework can be used to rule out other approaches for analysis, and then to subsequently improve PSA studies by emphasizing good predictor selection for both the propensity score and outcome stages of analysis. By using this approach, researchers may be able to reduce the risk of inducing bias, improving the credibility of causal estimates obtained using PSA.