Can We Reliably Measure Social Worker Communication Skills? Development of a Measure of Quality in Direct Child and Family Social Work Practice

Background and Purpose:

Communication skills are considered fundamental to child and family social work practice, yet few attempts have been made to define and measure such skills. This paper describes the development of a system for rating seven key elements of child and family social work practice, using audio recordings of direct practice encounters and the testing of that system for reliability.  The Motivational Interviewing Treatment Integrity (MITI 3.1) code was adapted for the field of child and family social work and a further three domains were developed in consultation with key stakeholders. The ability to quantitatively categorise direct social work practice provides an important foundation for exploring both practice skills and outcomes.

 

Methods:

Development of the new domains and review of the existing ones involved focus group meetings with experienced practitioners, academics and other key figures. One of these included senior national figures, including England’s Chief Social Worker for Children and Families. A second involved primarily experienced practitioners. Expert seminars were also held with leading academics in this area. The MITI was then presented and discussed and the applicability of elements of the MITI critically considered for the context of social work and specifically child protection services. Through this process, in addition to four dimensions from the MITI (“collaboration”, “enhancing autonomy”, “evocation (of motivation)” and “empathy”) three new dimensions were identified – “focus on the child”, “clarity about concerns or reason for involvement”, and “purposefulness”. To test the reliability of these measures they were piloted on a sample of 133 audio recordings of direct practice with families involved with child protection services in London. Of the 133 recordings, a further random sample of 28 (21%) were coded by three trained researchers using MITI with the three new child services domains. Inter-rater reliability was assessed using Krippendorff's α and Intra-class correlation (ICC).

Results:

Analyses demonstrate the ability to reliably measure key elements of practice, with Krippendorff's α scores ranging from .461 (good) to .937 (excellent) and ICC ranging from .731 (good) to .967 (excellent) across the seven key skills.

Conclusions and Implications:

The ability to define and reliably measure key elements of good social work, as reported in this study, has important implications for policy and practice. Such a measure can be used to facilitate the development of professional communication skills, provide better education and assessment, and assist organisations aiming to deliver and support high standards of direct practice.  However, the reliable coding of social work communication skills can be best understood as part of a broader programme of work aimed at relating practice skills to outcomes for children and families. Reliability, as established in this study, provides a vital foundation for current research exploring the predictive validity of the measure.