Gesture and speech integration: an exploratory study of a man with aphasia
Cocks, N., Sautin, L., Kita, S. , Morgan, G. & Zlotowitz, S. (2009). Gesture and speech integration: an exploratory study of a man with aphasia. International Journal of Language and Communication Disorders, 44(5), pp. 795-804. doi: 10.1080/13682820802256965
Abstract
Background: In order to fully comprehend a speaker’s intention in everyday communication, we integrate information from multiple sources including gesture and speech. There are no published studies that have explored the impact of aphasia on iconic co-speech gesture and speech integration.
Aims: To explore the impact of aphasia on co-speech gesture and speech integration in one participant with aphasia (SR) and 20 age-matched control participants.
Methods & Procedures: SR and 20 control participants watched video vignettes of people producing 21 verb phrases in 3 different conditions, verbal only (V), gesture only (G) and verbal gesture combined (VG). Participants were required to select a corresponding picture from one of four alternatives: integration target, a verbal only match, a gesture only match, and an unrelated foil. The probability of choosing the integration target in the VG that goes beyond what is expected from the probabilities of choosing the integration target in V and G was referred to as multi-modal gain(MMG).
Outcomes & Results: SR obtained a significantly lower multi-modal gain score than the control participants (p<0.05). Error analysis indicated that in speech and gesture integration tasks, SR relied on gesture in order to decode the message, whereas the control participants relied on speech in order to decode the message. Further analysis
of the speech only and gesture only tasks indicated SR had intact gesture comprehension but impaired spoken word comprehension.
Conclusions & Implications: The results confirm findings by Records (1994) which reported that impaired verbal comprehension leads to a greater reliance on gesture to
decode messages. Moreover, multi-modal integration of information from speech and iconic gesture can be impaired in aphasia. The findings highlight the need for further exploration of the impact of aphasia on gesture and speech integration.
| Publication Type: | Article |
|---|---|
| Additional Information: | The definitive version is available at |
| Publisher Keywords: | Gesture, aphasia, multi-modal integration, comprehension, RECOGNITION, HAND |
| Subjects: | R Medicine > RC Internal medicine |
| Departments: | School of Health & Psychological Sciences > Language & Communication Science |
| SWORD Depositor: |
Metadata
MetadataActions (login required)
Admin Login