Background & Purpose: During the last decade, cultural competence (CC) has been identified as an integral part of best practices in behavioral health care. The U.S. Surgeon General's Report on Mental Health and the Freedom Commission on Mental Health have highlighted the need for culturally responsive services to overcome racial and ethnic disparities in access and utilization. As agencies respond to this need and more providers receive CC training and education, there is a need for an empirical basis to demonstrate its effectiveness. The paper is based on the evaluation of the PRIME (Partners Reaching to Improve Multicultural Effectiveness) trainings designed to improve effectiveness of behavioral health providers serving diverse cultures. The development of quantitative research design for the study highlighted ongoing theoretical issues involved in conceptualizing cultural competence. Researchers grappled with the challenge of asking meaningful questions without being limited to service provision to one particular cultural group and measuring how providers take account of cultural identity while respecting the uniqueness of their consumers.

Methods: Forty-six behavioral health professionals attended 18 training sessions spanning a period of nine months. The study was pre-test post-test design with data being collected at the beginning and end of the training cycle. Trainees completed self-report questionnaires composed of demographic questions, questions about populations served, a modified version of the Multicultural Awareness Knowledge and Skills Survey (MAKSS) and an organizational cultural competence scale. Trainees also completed the Cross-Cultural Adaptability Inventory (CCAI) scale at the beginning and end of the training. Participant observation and content analysis of trainee logs was also included in the evaluation.

Results: The sample was predominantly female, American White and heterosexual. Over half had attended multicultural trainings previously. According to the modified MAKSS scale, the trainees made significant improvements overall, and on the knowledge and skills subscales. Trainees did not improve according to the MAKSS awareness subscale, which showed low reliability. Linear regression found no significant differences in terms of race, age or gender in predicting MAKSS scores, but attendance at previous trainings did correlate with higher scores. Overall, trainees did not show improvement on the CCAI score, but closer analysis found that trainees split into the following groups: negative change, positive change and no change. Gender was found to predict a decrease in scores. This was supplemented by the qualitative assessment, which found that changes in CC were non-linear.

Implications: The study demonstrated the ongoing limitations of existing CC measures. Conceptual issues that impact reliability and validity and the reliance on self-report mechanisms mean that these measures may not be able to capture CC in the workplace. Also, restricting measurement to pre- and post-training may not assess change accurately, as qualitative assessments indicate that the process of becoming cultural competent is not linear. Further work is needed to refine existing CC measures and broaden methods to capture the potential multifaceted aspect of acquiring CC.