| Multistep Deep System for Multimodal Emotion Detection With Invalid Data in the Internet of Things | |
Li, Minjia1; Xie, Lun1; Lv, Zeping2; Li, Juan3 ; Wang, Zhiliang1
| |
| Corresponding Author | Xie, Lun(xielun@ustb.edu.cn) |
| Abstract | The Internet of Things (IoT) technologies such as interconnection and edge computing help emotion recognition to be applied in healthcare, smart education, etc. However, the acquisition and transmission processes may have some situations, such as lost signals and serious interference noise caused by motion, which affect the quality of the received data and limit the performance of IoT emotion detection. We collectively refer to these as invalid data. A multi-step deep (MSD) system is proposed to reliably detect multimodal emotion by the collected records containing invalid data. Semantic compatibility and continuity are utilized to filter out the invalid data. The feature from invalid modal data is replaced through the imputation method to compensate for the impact of invalid data on emotion detection. In this way, the proposed system can automatically process invalid data and improve the recognition performance. Furthermore, considering the spatiotemporal information, the features of video and physiological signals are extracted by specific deep neural networks in the MSD system. The simulation experiments are conducted on a public multimodal database, and the performance of the MSD system measured by the unweighted average recall is better than that of the traditional system. The promising results observed in the experiments verify the potential influence of the proposed system in practical IoT applications. |
| Keyword | Feature extraction Internet of Things Semantics Databases Task analysis Emotion recognition Biomedical monitoring Internet of Things multimodal emotion detection invalid data multi-step deep (MSD) system deep neural networks |
| 2020 | |
| Language | 英语 |
| DOI | 10.1109/ACCESS.2020.3029288 |
| Source Publication | IEEE ACCESS
 |
| ISSN | 2169-3536 |
| Volume | 8Pages:187208-187221 |
| Indexed By | SCI |
| Funding Project | National Key Research and Development Program of China[2018YFC2001700] ; National Natural Science Foundation of China (Normal Project)[61672093] ; Beijing Municipal Natural Science Foundation[L192005] |
| Publisher | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| WOS Keyword | HEALTH-CARE IOT ; COMPUTATION |
| WOS Research Area | Computer Science ; Engineering ; Telecommunications |
| WOS Subject | Computer Science, Information Systems ; Engineering, Electrical & Electronic ; Telecommunications |
| WOS ID | WOS:000583561000001 |
| Citation statistics | |
| Document Type | 期刊论文 |
| Identifier | http://ir.psych.ac.cn/handle/311026/33374 |
| Collection | 认知与发展心理学研究室 |
| Corresponding Author | Xie, Lun |
| Affiliation | 1.Univ Sci & Technol Beijing, Sch Comp & Commun Engn, Beijing 100083, Peoples R China 2.Rehabil Hosp, Natl Rehabil Auxiliary Ctr, Beijing 100176, Peoples R China 3.Chinese Acad Sci, Inst Psychol, Beijing 100101, Peoples R China |
| Recommended Citation GB/T 7714 | Li, Minjia,Xie, Lun,Lv, Zeping,et al. Multistep Deep System for Multimodal Emotion Detection With Invalid Data in the Internet of Things[J]. IEEE ACCESS,2020,8:187208-187221. |
| APA | Li, Minjia,Xie, Lun,Lv, Zeping,Li, Juan,&Wang, Zhiliang.(2020).Multistep Deep System for Multimodal Emotion Detection With Invalid Data in the Internet of Things.IEEE ACCESS,8,187208-187221. |
| MLA | Li, Minjia,et al."Multistep Deep System for Multimodal Emotion Detection With Invalid Data in the Internet of Things".IEEE ACCESS 8(2020):187208-187221. |
| Files in This Item: | There are no files associated with this item. | |||||
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment