文章基本信息

标题：Impact of Affective Multimedia Content on the Electroencephalogram and Facial Expressions
本地全文：下载
作者：Siddharth Siddharth ; Tzyy-Ping Jung ; Terrence J. Sejnowski 等
期刊名称：Scientific Reports
电子版ISSN：2045-2322
出版年度：2019
卷号：9
期号：1
页码：1-10
DOI：10.1038/s41598-019-52891-2
出版社：Springer Nature
摘要：Most of the research in the field of affective computing has focused on detecting and classifying human emotions through electroencephalogram (EEG) or facial expressions. Designing multimedia content to evoke certain emotions has been largely motivated by manual rating provided by users. Here we present insights from the correlation of affective features between three modalities namely, affective multimedia content, EEG, and facial expressions. Interestingly, low-level Audio-visual features such as contrast and homogeneity of the video and tone of the audio in the movie clips are most correlated with changes in facial expressions and EEG. We also detect the regions associated with the human face and the brain (in addition to the EEG frequency bands) that are most representative of affective responses. The computational modeling between the three modalities showed a high correlation between features from these regions and user-reported affective labels. Finally, the correlation between different layers of convolutional neural networks with EEG and Face images as input provides insights into human affection. Together, these findings will assist in (1) designing more effective multimedia contents to engage or influence the viewers, (2) understanding the brain/body bio-markers of affection, and (3) developing newer brain-computer interfaces as well as facial-expression-based algorithms to read emotional responses of the viewers.