Magnitude of Task-Sampling Variability in Performance Assessment: A Meta-Analysis

Research output: Contribution to journalArticle

11 Citations (Scopus)

Abstract

This study examined the percentage of task-sampling variability in performance assessment via a meta-analysis. In total, 50 studies containing 130 independent data sets were analyzed. Overall results indicate that the percentage of variance for (a) differential difficulty of task was roughly 12% and (b) examinee's differential performance of the same task was approximately 26%. Based on moderator analysis, research design and subject area were significant predictors of variance component for tasks (σ2 t) and variance component for person-task interaction (σ2 pt). Mathematics and listening were associated with relatively high proportion of task-sampling variability whereas foreign language was least affected by this variability. To reduce task variation, some techniques are recommended: (a) incorporating more facets when possible, (b) using a crossed rather than a nested design, and (c) incorporating occasion as a facet.

Original languageEnglish
Pages (from-to)887-912
Number of pages26
JournalEducational and Psychological Measurement
Volume69
Issue number6
DOIs
Publication statusPublished - 2009 Jan 1

All Science Journal Classification (ASJC) codes

  • Education
  • Developmental and Educational Psychology
  • Applied Psychology
  • Applied Mathematics

Fingerprint Dive into the research topics of 'Magnitude of Task-Sampling Variability in Performance Assessment: A Meta-Analysis'. Together they form a unique fingerprint.

  • Cite this