文章基本信息

标题：Automated Essay Scoring Using Transformer Models
本地全文：下载
作者：Sabrina Ludwig ; Christian Mayer ; Christopher Hansen 等
期刊名称：Psych
电子版ISSN：2624-8611
出版年度：2021
卷号：3
期号：4
页码：897-915
DOI：10.3390/psych3040056
语种：English
出版社：MDPI AG
摘要：Automated essay scoring (AES) is gaining increasing attention in the education sector as it significantly reduces the burden of manual scoring and allows ad hoc feedback for learners. Natural language processing based on machine learning has been shown to be particularly suitable for text classification and AES. While many machine-learning approaches for AES still rely on a bag of words (BOW) approach, we consider a transformer-based approach in this paper, compare its performance to a logistic regression model based on the BOW approach, and discuss their differences. The analysis is based on 2088 email responses to a problem-solving task that were manually labeled in terms of politeness. Both transformer models considered in the analysis outperformed without any hyperparameter tuning of the regression-based model. We argue that, for AES tasks such as politeness classification, the transformer-based approach has significant advantages, while a BOW approach suffers from not taking word order into account and reducing the words to their stem. Further, we show how such models can help increase the accuracy of human raters, and we provide a detailed instruction on how to implement transformer-based models for one’s own purposes.