文章基本信息

标题：Performance Comparison of Natural Language Understanding Engines in the Educational Domain
本地全文：下载
作者：Victor Juan Jimenez Flores ; Oscar Juan Jimenez Flores ; Juan Carlos Jimenez Flores 等
期刊名称：International Journal of Advanced Computer Science and Applications(IJACSA)
印刷版ISSN：2158-107X
电子版ISSN：2156-5570
出版年度：2020
卷号：11
期号：8
DOI：10.14569/IJACSA.2020.0110892
出版社：Science and Information Society (SAI)
摘要：Recently, chatbots are having a great importance in different domains and are becoming more and more common in customer service. One possible cause is the wide variety of platforms that offer the natural language understanding as a service, for which no programming skills are required. Then, the problem is related to which platform to use to develop a chatbot in the educational domain. Therefore, the main objective of this paper is to compare the main natural language understanding (NLU) engines and determine which could perform better in the educational domain. In this way, researchers can make more justified decisions about which NLU engine to use to develop an educational chatbot. Besides, in this study, six NLU platforms were compared and performance was measured with the F1 score. Training data and input messages were extracted from Mariateguino Bot, which was the chatbot of the Jose´ Carlos Mari´ategui University during 2018. The results of this comparison indicates that Watson Assistant has the best performance, with an average F1 score of 0.82, which means that it is able to answer correctly in most cases. Finally, other factors can condition the choice of a natural language understanding engine, so that ultimately the choice is left to the user.
关键词：Chatbot; natural language understanding; NLU; F1 score; performance