Assessing International Labor Conditions: Using a Formative Simulation for Testing a Methodology
Method. The researchers used a controlled simulation to implement the application of the methodology. They recruited three teams, composed of three experts in international labor, to seek to implement the methodology, with each team addressing labor conditions in a single developing country. The members of the team independently evaluated a number of indicators for their assigned country. For example, each of the teams assessed 38 separate indicators for “freedom of association” and other indicators for acceptable conditions of work, discrimination, and forced or compulsory labor. These independent ratings were correlated by the research team. Each team was then brought together for a two day meeting in which they discussed their ratings and, when there were discrepancies, sought to resolve them. These meetings were recorded and analyzed to identify issues associated with the rating process and the bases for the differences in their assessments.
Results. The simulation uncovered a number of unanticipated problems with the methodology, These included problems in operationalization/definitions of the indicators, issues related to the structure of the evaluative framework, and problems with the restrictions on information sources. The researchers presented their findings and proposed specific modifications at a public meeting convened by the Department of Labor’s Bureau of Labor of International Labor Affairs.
Conclusions. The study demonstrated the utility of the formative simulation for an in-depth assessment of the NAS methodology and for identifying needed changes. It revealed strengths and weakness of the monitoring protocol. Beyond the conclusions about the specific methodology, the findings highlight the importance of developing realistic goals for assessing labor conditions in other countries. It also emphasized the importance of “on-the-ground” information for judging compliance with international labor standards.
1 National Research Council (2004). Monitoring International Labor Standards: Techniques and Sources of Information. Washington, DC: The National Academies Press.