文章基本信息

标题：BERT_SE: A Pre-Trained Language Representation Model for Software Engineering
本地全文：下载
作者：Eliane Maria De Bortoli Fávero ; Dalcimar Casanova
期刊名称：Computer Science & Information Technology
电子版ISSN：2231-5403
出版年度：2021
卷号：11
期号：19
语种：English
出版社：Academy & Industry Research Collaboration Center (AIRCC)
摘要：The application of Natural Language Processing (NLP) has achieved a high level of relevance in several areas. In the field of software engineering (SE), NLP applications are based on the classification of similar texts (e.g. software requirements), applied in tasks of estimating software effort, selection of human resources, etc. Classifying software requirements has been a complex task, considering the informality and complexity inherent in the texts produced during the software development process. The pre-trained embedding models are shown as a viable alternative when considering the low volume of textual data labeled in the area of software engineering, as well as the lack of quality of these data. Although there is much research around the application of word embedding in several areas, to date, there is no knowledge of studies that have explored its application in the creation of a specific model for the domain of the SE area. Thus, this article presents the proposal for a contextualized embedding model, called BERT_SE, which allows the recognition of specific and relevant terms in the context of SE. The assessment of BERT_SE was performed using the software requirements classification task, demonstrating that this model has an average improvement rate of 13% concerning the BERT_base model, made available by the authors of BERT. The code and pre-trained models are available at https://github.com/elianedb.
关键词：Word embedding;Software engineering;Domain-specific Model;Contextualized pre-trained model;BERT