Avatars are all around us: they represent real people online and colonise new worlds in the movies. In science, their role has been more limited. But avatars can be extremely useful in linguistics, new research shows. Scientists from the Max Planck Institute for Psycholinguistics use virtual avatars to investigate how real people behave in interaction. The method makes it possible to study with great precision how people adjust to each other in conversation.
An exciting question in psycholinguistics is how people adapt their speech to each other in conversation. The research method of choice has long been the “confederate”: a conversational partner who, without the other participant knowing, has been instructed to speak in certain ways. However, some aspects of speaking cannot be studied in this way, for instance minute changes in speech rate or intonation. Evelien Heyselaar, Peter Hagoort and Katrien Segaert have exchanged the human confederate for a virtual reality avatar, opening up exciting new possibilities for studying the dynamics of dialogue.
Best of both worlds
To get around the problems of human confederates, some studies have used pre-recorded sound files instead of confederates. However, speaking to a sound recording can feel very different from speaking in a face-to-face context, and indeed studies have shown that language behaviour is different when interacting with an sound recording compared to a video. Heyselaar and colleagues set out to determine whether virtual reality could be used to combine the best of the confederate and sound-only methods.
Virtual reality makes it possible to create a physical presence, an avatar, whose behaviour and speech patterns we can control to the tiniest detail. Heyselaar and colleagues decided to test the effectiveness of this avatar using syntactic priming, a phenomenon in which participants adapt their grammar usage to match that of the other. Because syntactic priming is well understood, it provides the ideal testing ground for an implementation using virtual reality.
Human-like avatars
Recent work has suggested that the degree to which people adapt can be influenced by their opinion of the person they are interacting with. Heyselaar and colleagues invited people to do a card game with either a human confederate, a human-like avatar, or a computer-like avatar. The two avatars looked identical and differed only in their behaviour: the human-like avatar showed realistic facial expressions such as smiles, eyebrow raises and eye contact, while the computer-like avatar had none of these.
As predicted, the human-like avatar worked just as well as the human confederate: both resulted in the same amount of grammatical adaptation, whereas the computer-like avatar did not. These results suggest that future conversational experiements can use virtual reality avatars and investigate behaviour in ways that were previously impossible.
Source: Evelien Heyselaar – Max Planck Institute
Image Source: The image is credited to MPI for Psycholinguistics, Heyselaar, Hagoort & Segaert
Original Research: Full open access research for “In dialogue with an avatar, language behavior is identical to dialogue with a human partner” by Evelien Heyselaar , Peter Hagoort, and Katrien Segaert in Biological Psychiatry. Published online December 16 2015 doi:10.3758/s13428-015-0688-7
Abstract
In dialogue with an avatar, language behavior is identical to dialogue with a human partner
The use of virtual reality (VR) as a methodological tool is becoming increasingly popular in behavioral research as its flexibility allows for a wide range of applications. This new method has not been as widely accepted in the field of psycholinguistics, however, possibly due to the assumption that language processing during human-computer interactions does not accurately reflect human-human interactions. Yet at the same time there is a growing need to study human-human language interactions in a tightly controlled context, which has not been possible using existing methods. VR, however, offers experimental control over parameters that cannot be (as finely) controlled in the real world. As such, in this study we aim to show that human-computer language interaction is comparable to human-human language interaction in virtual reality. In the current study we compare participants’ language behavior in a syntactic priming task with human versus computer partners: we used a human partner, a human-like avatar with human-like facial expressions and verbal behavior, and a computer-like avatar which had this humanness removed. As predicted, our study shows comparable priming effects between the human and human-like avatar suggesting that participants attributed human-like agency to the human-like avatar. Indeed, when interacting with the computer-like avatar, the priming effect was significantly decreased. This suggests that when interacting with a human-like avatar, sentence processing is comparable to interacting with a human partner. Our study therefore shows that VR is a valid platform for conducting language research and studying dialogue interactions in an ecologically valid manner.
“In dialogue with an avatar, language behavior is identical to dialogue with a human partner” by Evelien Heyselaar , Peter Hagoort, and Katrien Segaert in Biological Psychiatry. Published online December 16 2015 doi:10.3758/s13428-015-0688-7
