Assessing unidimensionality of polytomous data

This study investigated the performance of Poly-DIMTEST (PD) to assess unidimensionality of test data produced by polytomous items. Two types of polytomous data were considered: (1) tests in which all items had the same number of response categories, and (2) tests in which items had a mixed number of response categories. Test length, sample size, and the type of correlation matrix (used in factor analysis for selecting AT1 subset items) were varied in Type I error analyses. For the power study, the correlation between θs and the item-θ loadings were also varied. The results showed that PD was able to confirm unidimensionality for unidimensional simulated test data, with the average observed level of significance slightly below the nominal level. PD was also highly effective in detecting lack of unidimensionality in various two-dimensional tests. As expected, power increased as the sample size and test length increased, and the correlation between the θs decreased. The results also demonstrated that use of Pearson correlations to select AT1 items led to equally good or better performance than using polychoric correlations; therefore Pearson correlations are recommended for future use.