文章基本信息

标题：Hierarchical Multimodal Adaptive Fusion (HMAF) Network for Prediction of RGB-D Saliency
本地全文：下载
作者：Ying Lv ; Wujie Zhou
期刊名称：Computational Intelligence and Neuroscience
印刷版ISSN：1687-5265
电子版ISSN：1687-5273
出版年度：2020
卷号：2020
页码：1-9
DOI：10.1155/2020/8841681
出版社：Hindawi Publishing Corporation
摘要：Visual saliency prediction for RGB-D images is more challenging than that for their RGB counterparts. Additionally, very few investigations have been undertaken concerning RGB-D-saliency prediction. The proposed study presents a method based on a hierarchical multimodal adaptive fusion (HMAF) network to facilitate end-to-end prediction of RGB-D saliency. In the proposed method, hierarchical (multilevel) multimodal features are first extracted from an RGB image and depth map using a VGG-16-based two-stream network. Subsequently, the most significant hierarchical features of the said RGB image and depth map are predicted using three two-input attention modules. Furthermore, adaptive fusion of saliencies concerning the above-mentioned fused saliency features of different levels (hierarchical fusion saliency features) can be accomplished using a three-input attention module to facilitate high-accuracy RGB-D visual saliency prediction. Comparisons based on the application of the proposed HMAF-based approach against those of other state-of-the-art techniques on two challenging RGB-D datasets demonstrate that the proposed method outperforms other competing approaches consistently by a considerable margin.