Methods: After providing an overview of what YARH is, we will describe how Mathematica partnered with each YARH grantee to provide evaluation-related technical assistance. Evaluation technical assistance included helping grantees identify their target populations, develop theories of change and logic models, and learn about their target populations. During the second phase of YARH, dedicated liaisons from Mathematica worked with grantees on “usability tests” or small tests of very specific pieces of the intervention to make sure they worked. We supported grantees as they started using a CQI system to help monitor fidelity and other elements of the implementation, and they adjusted the models as needed. We worked with grantees to develop a formative evaluation to learn whether the interventions could be implemented as intended in the communities. We encouraged grantees to focus on the following questions:
- What do you hope to learn about your programs’ implementation?
- How will you know if the key components of your program are being delivered as intended?
- How will you know if your program is starting to show the intended effects?
Results: Grantees undertook an array of usability tests to determine the feasibility of specific elements of their models. Additionally, they conducted formative evaluations to understand what supports and structures were needed to implement their models with fidelity. Two of those grantees will shared what they learned through their formative evaluations.
Conclusions and implications: Providing a structure for grantees to learn about and test components of their interventions helped them strengthen their interventions and conduct formative evaluations that begin to show that they are having the desired effects.