其他摘要 | Rhythms are central to life and pervasive in human behaviors. The neural oscillations in our brains are rhythmic. Some typical forms of human movements (e.g., walking), termed biological motion (BM), also convey rhythmic signals. Traditionally, rhythm perception has been closely linked with auditory processing, and recent studies show that oscillatory brain activities encode complex rhythmic structures in auditory stimuli like speech and music. However, how neural oscillations encode rhythmic structures of visual BM and whether and to what extent auditory cues and auditory experience influence the visual processing of rhythmic BM signals remain largely unknown. Here we systematically investigated these issues in four studies.
In Study 1,we investigated the effect of temporal audiovisual correspondence on the perceptual processing of rhythmic BM. The results from four experiments showed that listening to the frequency-congruent footstep sounds, compared with incongruent or no-sound conditions, speeded up the visual search for point-light walker targets, with this effect driven by distinctive local motion cues (especially the accelerations in foot movement) independent of the global configuration of BM, suggesting a crossmodal integration mechanism triggered by specific kinematic features of BM that facilitate visual attention. An additional experiment revealed that the crossmodal attention facilitation occurred selectively for BM stimuli and increased with stimulus duration, indicating it may depend on the continuous tracking of audiovisual correspondences in BM signals.
In Study 2, we conducted five electroencephalogram (EEG) experiments to explore how rhythmic kinematic structures of human BM are dynamically represented in the brain and contribute to visual BM processing. We found that neural oscillations of observers entrained to the hierarchical rhythmic structures of walking and jumping-jack patterns (e.g., basic-level step-cycle and higher-level gait-cycle for walkers). Notably, only the cortical tracking of the higher-level rhythmic structure exhibited a BM processing specificity, when contrasted with the inverted control. This effect could be extended to different motion types and tasks, with its strength positively correlated with the perceptual sensitivity to BM stimuli. Modeling results further suggested that the neural encoding of spatiotemporally integrative cues (generated by the opponent motions of bilateral limbs) drives the selective cortical tracking effect. Moreover, the integrative cues mainly rely on the spatiotemporal summation of local motion cues rather than global configuration cues. Collectively, these findings underscored a cortical encoding mechanism based on periodic kinematic features of body movements, which underlies the dynamic construction of visual BM perception.
In Study 3, we further examined whether the cortical encoding of rhythmic signals underpins the audiovisual integration (AVI) of BM information (footstep sounds and walking motion) in two EEG experiments. The strength of cortical entrainment under the audiovisual condition significantly differed from the sum of the cortical tracking effect in the visual and auditory conditions at both step-cycle and gait-cycle frequencies, demonstrating a typical AVI effect. Furthermore, while audiovisual congruency enhanced the cortical tracking of rhythmic structures at both cycle frequencies relative to the incongruent condition, only the cortical tracking of the higher-level rhythmic structure (i.e., the gait-cycle) contributes to the specialized AVI process for BM and correlates with individuals' autistic traits. Together, these results demonstrated a hierarchical cortical entrainment process underlying the AVI of BM and associated with one's autistic traits.
Study 4 investigated the role of auditory experience on the visual encoding of the rhythmic structure in BM by comparing the cortical entrainment effect between the control group and the congenital deaf group adopting the same paradigm as in Study 2. Besides, we adopted non-biological visual stimuli with hierarchical rhythmic structures as controls to test the BM一specific effect. For BM stimuli, the cortical entrainment to higher-level rhythmic structures was decreased in the deaf group compared with the control group. For non-BM, the strength of cortical entrainment was not different between the two groups; however, the strongest neural response in the deaf group appeared in the auditory cortex rather than the visual cortex as in the control group. These findings indicate that the ability to encode rhythmic structures in visual BM partially depends on auditory experience, while the visual encoding of non-biological rhythmic stimuli may involve crossmodal compensation.
In conclusion, these studies demonstrated a hierarchical cortical entrainment process that underpins the perceptual processing and neural encoding of rhythmic BM. Crucially, the cortical encoding of kinematic cues embedded in higher-level rhythmic structures contributes to the specialized visual processing of BM, which is modulated by auditory BM cues and the observer's auditory experience. |
修改评论