认知与发展心理学研究室知识库

将看到的字与对应的读音整合起来对于阅读而言是一项重要的技能。阅读起始于字形和语音的结合，也就是形音整合。尽管形音整合有着重要的作用，但形音整合是否受到一些额外的训练的影响，例如音乐训练的影响还未有研究去探索。音乐训练是需要多种感觉参与的复杂的训练过程，对人的很多认知加工都有促进作用，例如多感觉整合。已有研究发现音乐训练对语言中的多感觉整合的加工有促进作用，但是目前的语言研究主要集中于考察言语的唇部动作与读音之间的视听整合。还没有研究考察长期的音乐训练是否会影响语言中的形音整合。已有对于形音整合的研究尽管考察了形音整合效应，但大多研究集中于一些拼音语言的字母水平，对汉字的研究还较少。另一方面，形音呈现时间的差异影响个体对于两个通道是否能够整合的判断。已有研究也发现了音乐训练对视听时间整合加工的影响，但对于形音时间整合加工是否受到长期的音乐训练的影响还是未知的。

本文主要通过四个实验来探索上述这两个问题。研究一采用行为实验与ERP实验，对音乐训练是否影响形音整合及其机制进行了考察。实验一的行为实验通过单独呈现形、单独呈现音以及视听同时呈现后形成形音一致、形音不一致的条件来研究形音整合效应。通过对比音乐家和非音乐家考察音乐训练是否对形音整合加工有促进作用。实验一的结果表明，音乐家不仅在两个单通道呈现的条件下比非音乐家有更快的反应，两种形音同时出现时反应也更快。实验二进一步通过ERP技术探究音乐训练影响形音整合的时间进程。结果发现在两侧头皮，音乐家在N1和晚期时间窗有一致性效应，且音乐家的N1效应与音乐训练年限存在相关。非音乐家只在晚期发现了一致性效应。结果表明音乐家在更早的时间窗口可以觉察到形音之间的一致性。在中线上两组没有差异。尽管在晚期窗口没有发现两组一致性效应的差异，但进一步精细的分析发现，音乐家的晚期效应有更早的降低。研究二考察了音乐训练对形音时间整合加工的影响。实验三通过考察音乐家和非音乐家在不同SOA下对不同速度呈现的形音的时间整合加工的差异。结果没有发现呈现速度上的差异，也没发现音乐家和非音乐家的差异。实验四通过操控形音之间的一致性对比了音乐家和非音乐家的形音时间整合加工。结果只发现了一致性的效应，形音不一致条件比一致条件有更小的时间整合窗口，并没有发现音乐家和非音乐家的差异。

综合研究一和研究二的结果，我们得出以下结论：音乐训练促进汉字中的形音整合加工。其内在机制可能是由于长期的音乐训练导致个体对形音之间是否一致有更早的觉察，因此促进了形音整合加工。但长期的音乐训练并没有促进个体的形音时间整合加工。不论有无音乐经验，个体在形音的时间整合加工中均不会受到刺激属性的影响，如呈现速度的影响，而受到形音之间的一致性影响。形音不一致会促进个体对形音时间不同步的判断。因此，在本研究的研究范式下，音乐训练只促进形音整合加工，但并不会影响形音时间整合加工。

The integration of visual inputs (e.g., letters/ characters) and corresponding speech sounds is crucial for reading. In spite of its importance, there is little knowledge about whether other life experiences would improve this integration process. Music training has been found to enhance multisensory integration in many ways, particularly audiovisual integration. So it is reasonable to test whether music training could affect the audiovisual integration in language. Moreover, music experience also influences the audiovisual temporal integration. But no studies have investigated the effect of music training on audiovisual temporal integration in written language.

In the present study, we explored these two questions by conducting four experiments. In experiment 1, we aim at testing whether long-term musical training might enhance audio-visual speech integration in Chinese using audio-only, visual-only, audiovisual congruent and incongruent stimuli. The results show that enhancement not only in musician`s ability to process audiovisual condition but also in unisensory condition. In experiment 2, we used event-related brain potential (ERP) to study the neural basic of this enhancement. Chinese characters were presented with congruent or incongruent speech sounds. For the musicians, we found that incongruent stimuli evoked a larger N1 and late time window than congruent stimuli in lateral side. However, we only found a late component for the nonmusicians. Besides, there was no difference between the two groups on that late component. But after a more detailed analysis, we found that the late component decreased earlier in musicians group. Correlation analyses showed that more years of music training is associated with larger N1 effect for the musicians. In experiment 3, we investigated whether music training influence audiovisual temporal integration in Chinese, especially when the stimuli presented with different speed. The results of experiment 3 show that there is no difference between musicians and nonmusicians, in both low and high speed. We also did not find any difference in two types of speed condition. In experiment 4, we then investigated whether music experience has a influence on detecting asynchrony of congruent and incongruent audiovisual information. We also did not find any difference between musicians and nonmusicians. Moreover, both groups show narrower temporal integration window in incongruent condition than congruent condition.

Our results thus indicate that musicians are better at integrating character-speech sound in Chinese. This is because musicians can detect character-speech sound incongruence earlier than nonmusicians. These results provide evidence that music training can help adults in integrating visual characters and auditory speech sounds in language. However, musicians do not have enhanced ability to detect asynchrony of character and speech sound, neither in different speed nor in different congruency. The results indicate that numerous years of music training may not influence audiovisual temporal integration in language.