City Research Online

Abstract

The sight and sound of a person speaking or a ball bouncing seem simultaneous, but their neural signals converge on different multimodal brain sites with different asynchronies. Brains come in different shapes and sizes, which further entails individual variations in this temporal distribution. How do people, and their brains, differ in how they integrate and synchronise hearing with vision? We measured temporal order judgements (TOJ) concurrently with the McGurk illusion (lip movements influence hearing), as functions of audiovisual asynchrony.

There was no correlation between McGurk susceptibility, the width of the window of integration, or the Just Noticeable Difference for temporal order discrimination (TOJ), suggesting distinct mechanisms. However each measure did correlate with the volume of distinct brain regions quantified from MR structural images (lIPS/rMTG, rMFG, and rMTG, respectively). Different measures also correlated with reading aloud versus listening comprehension. Counterintuitively, individual differences in the asynchrony for maximal McGurk and also Stream-Bounce illusions (sounds influence perceived visual motion) correlated negatively with Point of Subjective Simultaneity for TOJ: individuals who needed auditory lag for optimal integration, needed an auditory lead to perceive the same stimuli as simultaneous, and vice-versa. We propose that asynchronies within individual brain mechanisms are perceived relative to the average asynchrony across mechanisms.

These observations allow us to (1) fractionate 'multisensory perception' functionally and structurally into distinct mechanisms, (2) relate a profile of simple perceptual abilities to more complex cognitive abilities, and (3) explain how we can perceive the multisensory world accurately on average, despite evident disunity of audiovisual timing.

Publication Type:	Article
Additional Information:	Copyright Sage 2012
Subjects:	B Philosophy. Psychology. Religion > BF Psychology
Departments:	School of Health & Psychological Sciences > Psychology

Export

Downloads