Developing a Measure of Financial Knowledge from Survey Items Using Item Response Theory

Background

Financial capability for all has been identified as one of the grand challenges for social work (Uehara et al., 2014),which includes both financial knowledge and financial inclusion (Sherraden, 2013). Financial knowledge, which refers to people’s understanding of various components of everyday finances (Leskinen & Raijas, 2006), has not been carefully studied from a measurement perspective. Most empirical studies used survey items to measure people’s financial knowledge with a simple summary scale (Babiarz & Robb, 2014; Xiao et al., 2014). There is little research on the reliability and validity of these survey items. Besides, while measuring people’s financial knowledge, most studies did not consider people’s socioeconomic context particularly the income disparity. In this study, we examined three critical measurement issues: the reliability and validity of survey items and the measurement uniformity (Penfield & Camilli, 2007) across income groups.

Method

We used data from the 2009 and 2014 nationally representative Canadian Financial capability Survey (CFCS) (N=22,204). The CFCS included 14 objective assessment items to measure people's financial knowledge. We used Item Response Theory (IRT) to create a measure of financial knowledge from the original survey items and tested its reliability and measurement uniformity across income groups. Spearman's rank correlations were analyzed to test the construct validity.

Results

We ran a one-parameter logistic (1PL) model on 14 survey items (item 1-14). The item difficulty (b) levels ranged between -2.42 and 1.30 but six items (item 1, 6, 9, 11, 12, and 13) were redundant. We then ran the model on the remaining eight items and the item difficulty levels then ranged between -2.41 and 1.39. No item showed redundant at this point but item 5 and item 10 did not qualify the uniformity test across income groups. Therefore, the final scale included six items (item 2, 3, 4, 7, 8, and 14). The item difficulty levels of which ranged between -2.47 and .96. The results of the Test Information Function (TIF) showed that the scale is reliable between the difficulty levels of b = -3 and b = 2, and the scale provides the maximum information at the difficulty level of b = -1.75.

The convergent validity test showed that the financial knowledge scale is positively correlated with account balance checking (ρ= 0.14, p<0.001), saving (ρ= 0.14, p<0.001), and investment (ρ= 0.16, p<0.001 respectively. The divergent validity test also showed the expected negative correlations between financial knowledge scale and late payment of bills (ρ = -0.03, p<0.001) and seeking-no-financial advice respectively (ρ = -0.19, p<0.001).

Implications

The 14 items in the CFCS have some redundancy when used as a summary scale. We suggest that using the summary scale may be inefficient, and, in addition, does not uniformly measure financial knowledge across income groups. Our revised financial knowledge scale with six items demonstrated suitable reliability and validity. We argue this revised scale can be used to measure financial knowledge across income groups, which has important implications for social work researchers studying financial practices and financial capability.