PSYCH OpenIR  > 认知与发展心理学研究室
Multistep Deep System for Multimodal Emotion Detection With Invalid Data in the Internet of Things
Li, Minjia1; Xie, Lun1; Lv, Zeping2; Li, Juan3; Wang, Zhiliang1
Corresponding AuthorXie, Lun(
AbstractThe Internet of Things (IoT) technologies such as interconnection and edge computing help emotion recognition to be applied in healthcare, smart education, etc. However, the acquisition and transmission processes may have some situations, such as lost signals and serious interference noise caused by motion, which affect the quality of the received data and limit the performance of IoT emotion detection. We collectively refer to these as invalid data. A multi-step deep (MSD) system is proposed to reliably detect multimodal emotion by the collected records containing invalid data. Semantic compatibility and continuity are utilized to filter out the invalid data. The feature from invalid modal data is replaced through the imputation method to compensate for the impact of invalid data on emotion detection. In this way, the proposed system can automatically process invalid data and improve the recognition performance. Furthermore, considering the spatiotemporal information, the features of video and physiological signals are extracted by specific deep neural networks in the MSD system. The simulation experiments are conducted on a public multimodal database, and the performance of the MSD system measured by the unweighted average recall is better than that of the traditional system. The promising results observed in the experiments verify the potential influence of the proposed system in practical IoT applications.
KeywordFeature extraction Internet of Things Semantics Databases Task analysis Emotion recognition Biomedical monitoring Internet of Things multimodal emotion detection invalid data multi-step deep (MSD) system deep neural networks
Source PublicationIEEE ACCESS
Indexed BySCI
Funding ProjectNational Key Research and Development Program of China[2018YFC2001700] ; National Natural Science Foundation of China (Normal Project)[61672093] ; Beijing Municipal Natural Science Foundation[L192005]
WOS Research AreaComputer Science ; Engineering ; Telecommunications
WOS SubjectComputer Science, Information Systems ; Engineering, Electrical & Electronic ; Telecommunications
WOS IDWOS:000583561000001
Citation statistics
Document Type期刊论文
Corresponding AuthorXie, Lun
Affiliation1.Univ Sci & Technol Beijing, Sch Comp & Commun Engn, Beijing 100083, Peoples R China
2.Rehabil Hosp, Natl Rehabil Auxiliary Ctr, Beijing 100176, Peoples R China
3.Chinese Acad Sci, Inst Psychol, Beijing 100101, Peoples R China
Recommended Citation
GB/T 7714
Li, Minjia,Xie, Lun,Lv, Zeping,et al. Multistep Deep System for Multimodal Emotion Detection With Invalid Data in the Internet of Things[J]. IEEE ACCESS,2020,8:187208-187221.
APA Li, Minjia,Xie, Lun,Lv, Zeping,Li, Juan,&Wang, Zhiliang.(2020).Multistep Deep System for Multimodal Emotion Detection With Invalid Data in the Internet of Things.IEEE ACCESS,8,187208-187221.
MLA Li, Minjia,et al."Multistep Deep System for Multimodal Emotion Detection With Invalid Data in the Internet of Things".IEEE ACCESS 8(2020):187208-187221.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Li, Minjia]'s Articles
[Xie, Lun]'s Articles
[Lv, Zeping]'s Articles
Baidu academic
Similar articles in Baidu academic
[Li, Minjia]'s Articles
[Xie, Lun]'s Articles
[Lv, Zeping]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Li, Minjia]'s Articles
[Xie, Lun]'s Articles
[Lv, Zeping]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.