文章基本信息

标题：Factors Affecting Untrained Raters’ Cognition in Rating Oral Proficiency of English Language Learners: Insights from Iran
本地全文：下载
作者：Nasimeh Nouhi Jadesi ; Alireza Ahmadi ; Seyyed Ayatollah Razmjoo 等
期刊名称：Applied Research on English Language
印刷版ISSN：2252-0198
电子版ISSN：2322-5343
出版年度：2022
卷号：11
期号：2
页码：1-24
DOI：10.22108/are.2021.129400.1744
语种：English
出版社：University of Isfahan
摘要：Score decision-making is largely an undocumented process in performance assessment. To conduct a more in-depth cognitive study in scoring, one must ask if these underlying processes can be identified efficiently and objectively. To this end, the present study attempted to shed some light on how Iranian teachers as untrained raters rate speech samples of learners and how their cognition functions in the decision-making process in terms of the scores they assign. A series of monologues were obtained from a group of language learners; afterward, English language teachers were asked to rate them. The raters were asked both to assign a score and provide comments regarding why they assigned a specific score. Having rated the samples, the raters were individually interviewed. The results of the recorded interviews and the comments they had provided on scores were subjected to qualitative analysis like coding and extracting both idiosyncratic and shared features of the raters’ cognition. The results revealed that some of the factors attended to by the raters were both linguistic and relevant to speaking proficiency construct like fluency, accuracy, and complexity. Some other factors influencing the raters while rating were non-linguistic and not directly related to speaking proficiency construct like the tone of voice, personality feature of the testee, etc. It seemed that the untrained raters did not have a clear definition of oral proficiency construct. The implications of the study for rater training programs have been discussed.