文章基本信息

标题：Toward a better judgment of item relevance in progress testing
本地全文：下载
作者：Xandra M. C. Janssen-Brandt ; Arno M. M. Muijtjens ; Dominique M. A. Sluijsmans 等
期刊名称：BMC Medical Education
印刷版ISSN：1472-6920
出版年度：2017
卷号：17
期号：1
页码：151-158
DOI：10.1186/s12909-017-0989-x
出版社：BioMed Central
摘要：Background Items must be relevant to ensure item quality and test validity. Since “item relevance” has not been operationalized yet, we developed a rubric to define it. This study explores the influence of this rubric on the assessment of item relevance and on inter-rater agreement. Methods Members of the item review committee (RC) and students, teachers, and alumni (STA) reassessed the relevance of 50 previously used progress test (PT) items and decided about their inclusion using a 5-criteria rubric. Data were analyzed at item level using paired samples t-tests, Intraclass Correlation Coefficients (ICC), and linear regression analysis, and at rater level in a generalizability analysis per group. Results The proportion of items that the RC judged relevant enough to be included decreased substantially from 1.00 to 0.72 ( p 0.7 across items. The relation between inclusion and relevance was strong (correlation = 0.89, p Conclusions Use of the rubric results in a stricter evaluation of items’ appropriateness for inclusion in the PT and facilitates agreement between the RC and other stakeholders. Hence, it may help increase the acceptability and validity of the PT.
关键词：Item relevance ; Midwifery education ; Progress testing ; Rubric