认知与发展心理学研究室知识库

日常生活中，人们接触到的事物和情景一般同时包含多种模态的信息(如声音、图像等)，人脑会对来自不同感觉通道(如听觉、视觉等)的信息进行比较、评估以及整合，这种多通道信息的整合使人对外部世界形成了统一、连贯、稳定的认知。

研究发现，多通道信息整合可以在视觉信息处于无意识条件下发生，但是以往研究主要集中于基于经验的语义关联和时空一致的多通道刺激在具体水平上的信息整合，而对于多通道信息在抽象水平上的无意识整合的研究较少。因此，本研究结合异同判断范式和后摄掩蔽范式，考察内隐多通道信息整合是否可以发生在抽象水平上以及刺激区分度对其整合的影响。

本研究系统考察多通道间抽象信息的内隐整合及影响因素。研究一考察在视觉信息处于无意识条件时，视听抽象信息内隐整合的可能性。结果发现，个体可以无意识地习得视听信息间的抽象关系，表现为视听信息关系不一致比视听信息间关系一致降低对视觉目标刺激的反应正确率。研究二进一步考察任务相关刺激和任务无关刺激的异同判断难度如何影响抽象视听信息的内隐整合。结果发现，无关刺激和目标刺激的区分度影响抽象视听信息的内隐整合。当任务相关刺激与任务无关刺激较难分辨时，没有显著的视听一致性效应;当任务相关刺激较难分辨而任务无关刺激较易分辨时，发现了对视觉目标刺激反应的多通道促进效应，对听觉目标刺激反应的多通道抑制效应;当任务相关刺激和任务无关刺激均较易分辨时，仅在听觉刺激作为任务无关刺激时，发现了视听一致性效应，说明存在多通道信息内隐整合的不对称性。

本研究探讨抽象水平的视听信息内隐整合，不仅有助于加深对人脑表征多模态信息方式的认识，还有助于加强对意识和注意与信息整合关系的理解。同时，可以为现代多媒体教学方法的改善提供依据，并为教育应用软件的设置以及人工智能中相关算法的改进提供启示。

In daily life, things and scenes that people come into contact generally contain multiple forms of modal information (such as sounds and images). These information acts on different sensory organs, and human brain will compare, integrate, and evaluate the information from different sensory channels (such as auditory and visual modality).The integration of the multisensory information has led to a unified, coherent, and stable understanding of the external world.

Some studies have found that multisensory integration can occur when the visual information is at the unconscious status. However, previous integration studies mainly focused on the experience-based semantic association and concrete information integration of multimodal stimuli which were consistent with time and space. It remains unclear whether the abstract relation of multi-sensory information can be unconsciously integrated. Therefore, this study was aimed to combine a same-different paradigm and a backward masking paradigm to investigate whether implicit multi-sensory integration can occur at the abstract level and the factors which could influence the integration.

This study systematically investigates the implicit multi-sensory integration of abstract information and its influencing factors. The first study was aimed to explore whether people can implicitly or unconsciously integrate the abstract relation of the audio-visual information when the visual information was at the subliminal level. The results showed that the inconsistent relation between audio-visual information decreases the response accuracy of visual target stimuli compared with the consistent relation between audio-visual information, indicating that the abstract relation between the audio-visual information could be implicitly integrated. In the second study, the effect of the discrimination difficulty of the target and the task-irrelevant stimuli on implicit integration of the abstract audio-visual information was further investigated.The results showed that the discrimination difficulty of the target and task-irrelevant stimuli influenced implicit abstract integration of the audio-visual information. There was no significant consistency effect when the discrimination of the target and irrelevant stimuli were both difficult. When the target stimuli were difficult to discriminate and the irrelevant stimuli were easy to discriminate, there was multisensory facilitation effect for the visual target stimuli but multisensory inhibitory effect for the auditory target stimuli. Further, when both target and irrelevant stimuli were easy to discriminate, there was a significant consistency effect only for the target visual stimuli, suggesting there was a modality asymmetry in implicit multisensory integration.

This study explored the implicit visual auditory integration at the abstract level, which can help to understand how the human brain integrates and characterizes the multimodal information and what the relationship between consciousness, attention, and information integration is. The findings give some insights for the improvement of modern multimedia teaching methods, and some inspirations for the setting of educational application software as well as for the improvement of related algorithms in artificial intelligence.