Could the Processes and Outcomes of a Program for Sex Trafficked Youth be Evaluated? Lessons from an Evaluability Assessment

Background and Purpose: A growing body of research documents the negative consequences of sex trafficking on children and youth. Yet, very little research exists on the provision of services to this population and there are hardly any evaluations of these programs. At the initial stages of an evaluation of the processes and outcomes of an Israeli program serving youth (ages 13-15) who are victims of domestic sex trafficking, we were confronted with population, program design and program implementation issues that called for an evaluabilty assessment. Our main goal was to assess whether an evaluation of the program is feasible and likely to provide useful information, and examine how can we best prepapre the program for the evaluation.

This paper presents the process and outcomes of the evaluability assessment, focusing on four unique evaluation challenges posed by a program for trafficked youth and the solutions we proposed to meet them.

Methods: The evaluated program is run by an NGO specializing in interventions with hard to reach youth at risk in collaboration of municipal and state social sevices. The evaluabilty assessment was conducted by the authors between August 2014 to August 2015. It aimed to accomplish the following tasks: 1. Studying the program history, intervention approach, design, and operation; 2. Observing the program in action; 3. Determining the program’s capacity for data collection, management, and analysis; 4. Clarifying the program's goals and objectives; and 5.Mapping and understanding the organizational context of the program and the relationships among its stakeholders. Data collection was based on relevant documents of, and about, the program; interviews and meetings with program's staff and stakeholders; and participant and non-participant observations of: relevant meetings, discussions, events, and of the program's facilities.

Findings: The analysis and conceptualization of the evaluability data raised four main challenges to the implementation of a systematic evaluation of an intervention program for sex-trafficked youth: a) Conceptualization - operationalizing the program's goal to "reduce damages and get youth out of the cycle of sex exploitation" as measurable outcomes; b) Measurement – developing and implementing valid and reliable instruments and procedures for learning about transient, suspicious and "chameleon-like" population; c) Ethics – avoiding the collection of "forbidden" evaluation data that could incriminate program clients; d) Politics-  navigating tensions between formal and informal intervention approaches held by program stakeholders.

Conclusion and Implications: The conceptual, methodological, ethical and political issues highlighted by the evaluability assessment presented seem to reflect the unique nature of services targeting  young people who have fallen through the cracks in society, through the use of both formal and informal intervention strategies. The challenges posed by such programs to evaluators may be overcome by adding to the evaluation a phase of evaluability assessment in which these challenges are identified and pragmatic solutions are provided by the evaluation team.