Crossmodal Language Comprehension-Psycholinguistic Insights and Computational Approaches | |
Alacam, O (Alacam, Ozge)1; Li, XS (Li, Xingshan)2; Menzel, W (Menzel, Wolfgang)1; Staron, T (Staron, Tobias)1 | |
第一作者 | Alacam, O (Alacam, Ozge) |
通讯作者邮箱 | alacam@informatik.uni-hamburg.de ; staron@informatik.uni-hamburg.de |
心理所单位排序 | 2 |
摘要 | Crossmodal interaction in situated language comprehension is important for effective and efficient communication. The relationship between linguistic and visual stimuli provides mutual benefit: While vision contributes, for instance, information to improve language understanding, language in turn plays a role in driving the focus of attention in the visual environment. However, language and vision are two different representational modalities, which accommodate different aspects and granularities of conceptualizations. To integrate them into a single, coherent system solution is still a challenge, which could profit from inspiration by human crossmodal processing. Based on fundamental psycholinguistic insights into the nature of situated language comprehension, we derive a set of performance characteristics facilitating the robustness of language understanding, such as crossmodal reference resolution, attention guidance, or predictive processing. Artificial systems for language comprehension should meet these characteristics in order to be able to perform in a natural and smooth manner. We discuss how empirical findings on the crossmodal support of language comprehension in humans can be applied in computational solutions for situated language comprehension and how they can help to mitigate the shortcomings of current approaches. |
关键词 | language comprehension crossmodality psycholinguistics incrementality prediction speaker intention |
2020-01 | |
语种 | 英语 |
DOI | 10.3389/fnbot.2020.00002 |
发表期刊 | FRONTIERS IN NEUROROBOTICS |
ISSN | 1662-5218 |
卷号 | 14页码:不详 |
期刊论文类型 | Review |
收录类别 | SCIE |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.psych.ac.cn/handle/311026/31317 |
专题 | 认知与发展心理学研究室 |
作者单位 | 1.Univ Hamburg, Dept Informat, Nat Language Syst Grp, Hamburg, Germany 2.Chinese Acad Sci, Inst Psychol, Reading & Visual Cognit Lab, Beijing, Peoples R China |
推荐引用方式 GB/T 7714 | Alacam, O ,Li, XS ,Menzel, W ,et al. Crossmodal Language Comprehension-Psycholinguistic Insights and Computational Approaches[J]. FRONTIERS IN NEUROROBOTICS,2020,14:不详. |
APA | Alacam, O ,Li, XS ,Menzel, W ,&Staron, T .(2020).Crossmodal Language Comprehension-Psycholinguistic Insights and Computational Approaches.FRONTIERS IN NEUROROBOTICS,14,不详. |
MLA | Alacam, O ,et al."Crossmodal Language Comprehension-Psycholinguistic Insights and Computational Approaches".FRONTIERS IN NEUROROBOTICS 14(2020):不详. |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
Crossmodal Language (800KB) | 期刊论文 | 出版稿 | 限制开放 | CC BY-NC-SA | 请求全文 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论