文章基本信息

标题：Deep multiple instance learning for foreground speech localization in ambient audio from wearable devices
本地全文：下载
作者：Rajat Hebbar ; Pavlos Papadopoulos ; Ramon Reyes 等
期刊名称：EURASIP Journal on Audio, Speech, and Music Processing
印刷版ISSN：1687-4714
电子版ISSN：1687-4722
出版年度：2021
卷号：2021
期号：1
页码：1
DOI：10.1186/s13636-020-00194-0
出版社：Hindawi Publishing Corporation
摘要：Over the recent years, machine learning techniques have been employed to produce state-of-the-art results in several audio related tasks. The success of these approaches has been largely due to access to large amounts of open-source datasets and enhancement of computational resources. However, a shortcoming of these methods is that they often fail to generalize well to tasks from real life scenarios, due to domain mismatch. One such task is foreground speech detection from wearable audio devices. Several interfering factors such as dynamically varying environmental conditions, including background speakers, TV, or radio audio, render foreground speech detection to be a challenging task. Moreover, obtaining precise moment-to-moment annotations of audio streams for analysis and model training is also time-consuming and costly. In this work, we use multiple instance learning (MIL) to facilitate development of such models using annotations available at a lower time-resolution (coarsely labeled). We show how MIL can be applied to localize foreground speech in coarsely labeled audio and show both bag-level and instance-level results. We also study different pooling methods and how they can be adapted to densely distributed events as observed in our application. Finally, we show improvements using speech activity detection embeddings as features for foreground detection.
关键词：Foreground speech detection ; Multiple instance learning ; Wearable audio ; Weakly labeled audio