Institutional Repository of Key Laboratory of Behavioral Science, CAS
Investigating inner properties of multimodal representation and semantic compositionality with brain-based componential semantics | |
Wang, Shaonan1,2; Zhang, Jiajun1,2; Lin, Nan3,4; Zong, Chengqing1,2,5 | |
2018 | |
会议名称 | 32nd AAAI Conference on Artificial Intelligence, AAAI 2018 |
会议录名称 | 32nd AAAI Conference on Artificial Intelligence, AAAI 2018 |
页码 | 5964-5972 |
会议日期 | February 2, 2018 - February 7, 2018 |
会议地点 | New Orleans, LA, United states |
出版者 | AAAI press |
产权排序 | 3 |
摘要 | Multimodal models have been proven to outperform text-based approaches on learning semantic representations. However, it still remains unclear what properties are encoded in multimodal representations, in what aspects do they outperform the single-modality representations, and what happened in the process of semantic compositionality in different input modalities. Considering that multimodal models are originally motivated by human concept representations, we assume that correlating multimodal representations with brain-based semantics would interpret their inner properties to answer the above questions. To that end, we propose simple interpretation methods based on brain-based componential semantics. First we investigate the inner properties of multimodal representations by correlating them with corresponding brain-based property vectors. Then we map the distributed vector space to the interpretable brain-based componential space to explore the inner properties of semantic compositionality. Ultimately, the present paper sheds light on the fundamental questions of natural language understanding, such as how to represent the meaning of words and how to combine word meanings into larger units. Copyright © 2018, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. |
关键词 | Compositionality - Input modalities - Interpretation methods - Learning semantics - Multimodal models - Natural language understanding - Text-based approach - Word meaning |
学科领域 | Semantics |
ISBN号 | 9781577358008 |
收录类别 | EI |
语种 | 英语 |
EI入藏号 | 20190506435859 |
EI主题词 | Artificial intelligence - Natural language processing systems - Vector spaces |
EI分类号 | 723.2 Data Processing and Image Processing - 723.4 Artificial Intelligence - 921 Mathematics |
文献类型 | 会议论文 |
条目标识符 | http://ir.psych.ac.cn/handle/311026/30040 |
专题 | 中国科学院行为科学重点实验室 |
作者单位 | 1.National Laboratory of Pattern Recognition, CASIA, Beijing, China; 2.University of Chinese Academy of Sciences, Beijing, China; 3.CAS Key Laboratory of Behavioural Science, Institute of Psychology, Beijing, China; 4.Department of Psychology, University of Chinese Academy of Sciences, Beijing, China; 5.CAS Center for Excellence in Brain Science and Intelligence Technology, Shanghai, China |
推荐引用方式 GB/T 7714 | Wang, Shaonan,Zhang, Jiajun,Lin, Nan,et al. Investigating inner properties of multimodal representation and semantic compositionality with brain-based componential semantics[C]:AAAI press,2018:5964-5972. |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
Wang et al 2018 AAAI(978KB) | 会议论文 | 限制开放 | CC BY-NC-SA | 请求全文 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论