文章基本信息

标题：Fine-Grained Human Evaluation of Neural Versus Phrase-Based Machine Translation
本地全文：下载
作者：Filip Klubička ; Antonio Toral ; Víctor M. Sánchez-Cartagena 等
期刊名称：The Prague Bulletin of Mathematical Linguistics
印刷版ISSN：0032-6585
电子版ISSN：1804-0462
出版年度：2017
卷号：108
期号：1
页码：121-132
DOI：10.1515/pralin-2017-0014
语种：English
出版社：Walter de Gruyter GmbH
摘要：We compare three approaches to statistical machine translation (pure phrase-based, factored phrase-based and neural) by performing a fine-grained manual evaluation via error annotation of the systems’ outputs. The error types in our annotation are compliant with the multidimensional quality metrics (MQM), and the annotation is performed by two annotators. Inter-annotator agreement is high for such a task, and results show that the best performing system (neural) reduces the errors produced by the worst system (phrase-based) by 54%.