Technology could lead to ‘empathetic’ robots and reactive on-line learning programmes.
Computers are able to read a person’s body language to tell whether they are bored or interested in what they see on the screen, according to a new study led by body language expert Dr Harry Witchel, Discipline Leader in Physiology at Brighton and Sussex Medical School.
The research shows that by measuring a person’s movements as they use a computer, it is possible to judge their level of interest by monitoring whether they display the tiny movements that people usually constantly exhibit, known as non-instrumental movements. If someone is absorbed in what they are watching or doing — what Dr Witchel calls ‘rapt engagement’ — there is a decrease in these involuntary movements.
Dr Witchel said: “Our study showed that when someone is really highly engaged in what they’re doing, they suppress these tiny involuntary movements. It’s the same as when a small child, who is normally constantly on the go, stares gaping at cartoons on the television without moving a muscle.”
The discovery could have a significant impact on the development of artificial intelligence. Future applications could include the creation of online tutoring programmes that adapt to a person’s level of interest, in order to re-engage them if they are showing signs of boredom. It could even help in the development of companion robots, which would be better able to estimate a person’s state of mind.
Also, for experience designers such as movie directors or game makers, this technology could provide complementary moment-by-moment reading of whether the events on the screen are interesting. While viewers can be asked subjectively what they liked or disliked, a nonverbal technology would be able to detect emotions or mental states that people either forget or prefer not to mention.
“Being able to ‘read’ a person’s interest in a computer programme could bring real benefits to future digital learning, making it a much more two-way process. Further ahead it could help us create more empathetic companion robots, which may sound very ‘sci fi’ but are becoming a realistic possibility within our lifetimes,” Dr Witchel said.
In the study, 27 participants faced a range of three-minute stimuli on a computer, from fascinating games to tedious readings from EU banking regulation, while using a handheld trackball to minimise instrumental movements, such as moving the mouse. Their movements were quantified over the three minutes using video motion tracking. In two comparable reading tasks, the more engaging reading resulted in a significant reduction (42%) of non-instrumental movement.
The study team also included two of Dr Witchel’s team, Carlos Santos and Dr James Ackah, a media expert from the University of Sussex Carina Westling, and the clinical biomechanics group at Staffordshire University led by Prof Nachiappan Chockalingam.
Source: BSMS
Image Source: The image is in the public domain.
Original Research: Abstract for “Non-Instrumental Movement Inhibition (NIMI) Differentially Suppresses Head and Thigh Movements during Screenic Engagement: Dependence on Interaction” by Harry J. Witchel, Carlos P. Santos, James K. Ackah, Carina E. I. Westling and Nachiappan Chockalingam in Frontiers in Psychology. Published online February 22 2016 doi:10.3389/fpsyg.2016.00157
Abstract
Non-Instrumental Movement Inhibition (NIMI) Differentially Suppresses Head and Thigh Movements during Screenic Engagement: Dependence on Interaction
Background: Estimating engagement levels from postural micromovements has been summarized by some researchers as: increased proximity to the screen is a marker for engagement, while increased postural movement is a signal for disengagement or negative affect. However, these findings are inconclusive: the movement hypothesis challenges other findings of dyadic interaction in humans, and experiments on the positional hypothesis diverge from it.
Hypotheses: (1) Under controlled conditions, adding a relevant visual stimulus to an auditory stimulus will preferentially result in Non-Instrumental Movement Inhibition (NIMI) of the head. (2) When instrumental movements are eliminated and computer-interaction rate is held constant, for two identically-structured stimuli, cognitive engagement (i.e., interest) will result in measurable NIMI of the body generally.
Methods: Twenty-seven healthy participants were seated in front of a computer monitor and speakers. Discrete 3-min stimuli were presented with interactions mediated via a handheld trackball without any keyboard, to minimize instrumental movements of the participant’s body. Music videos and audio-only music were used to test hypothesis (1). Time-sensitive, highly interactive stimuli were used to test hypothesis (2). Subjective responses were assessed via visual analog scales. The computer users’ movements were quantified using video motion tracking from the lateral aspect. Repeated measures ANOVAs with Tukey post hoc comparisons were performed.
Results: For two equivalently-engaging music videos, eliminating the visual content elicited significantly increased non-instrumental movements of the head (while also decreasing subjective engagement); a highly engaging user-selected piece of favorite music led to further increased non-instrumental movement. For two comparable reading tasks, the more engaging reading significantly inhibited (42%) movement of the head and thigh; however, when a highly engaging video game was compared to the boring reading, even though the reading task and the game had similar levels of interaction (trackball clicks), only thigh movement was significantly inhibited, not head movement.
Conclusions: NIMI can be elicited by adding a relevant visual accompaniment to an audio-only stimulus or by making a stimulus cognitively engaging. However, these results presume that all other factors are held constant, because total movement rates can be affected by cognitive engagement, instrumental movements, visual requirements, and the time-sensitivity of the stimulus.
“Non-Instrumental Movement Inhibition (NIMI) Differentially Suppresses Head and Thigh Movements during Screenic Engagement: Dependence on Interaction” by Harry J. Witchel, Carlos P. Santos, James K. Ackah, Carina E. I. Westling and Nachiappan Chockalingam in Frontiers in Psychology. Published online February 22 2016 doi:10.3389/fpsyg.2016.00157
