Everything We See Is a Mash-up of the Brain’s Last 15 Seconds of Visual Information

Summary: Researchers reveal how the brain creates an illusion of visual stability.

Source: The Conversation

Our eyes are continuously bombarded by an enormous amount of visual information – millions of shapes, colours and ever-changing motion all around us. For the brain, this is no easy feat.

On the one hand, the visual world alters continuously because of changes in light, viewpoint and other factors. On the other, our visual input constantly changes due to blinking and the fact that our eyes, head and body are frequently in motion.

To get an idea of the “noisiness” of this visual input, place a phone in front of your eyes and record a live video while you are walking around and looking at different things. The jittery, messy result is exactly what your brain deals with in every moment of your visual experience.

This can be seen also in the video below. The white circle on the right shows potential eye movements, and the blurry blob on the left reveals the jumpy visual input in every moment.

Credit: Sebastiaan Mathôt

Yet, seeing never feels like work for us. Rather than perceiving the fluctuations and visual noise that a video might record, we perceive a consistently stable environment. So how does our brain create this illusion of stability? This process has fascinated scientists for centuries and it is one of the fundamental questions in vision science.

The time machine brain

In our latest research, we discovered a new mechanism that, among others, can explain this illusory stability. The brain automatically smoothes our visual input over time. Instead of analysing every single visual snapshot, we perceive in a given moment an average of what we saw in the past 15 seconds. So, by pulling together objects to appear more similar to each other, our brain tricks us into perceiving a stable environment. Living “in the past” can explain why we do not notice subtle changes that occur over time.

In other words, the brain is like a time machine which keeps sending us back in time. It’s like an app that consolidates our visual input every 15 seconds into one impression so that we can handle everyday life. If our brains were always updating in real time, the world would feel like a chaotic place with constant fluctuations in light, shadow and movement. We would feel like we were hallucinating all the time.

We created an illusion to illustrate how this stabilisation mechanism works. Looking at the video below, the face on the left side slowly ages for 30 seconds, and yet, it is very difficult to notice the full extent of the change in age. In fact, observers perceive the face as ageing more slowly than it actually is.

To test this illusion we recruited hundreds of participants and asked them to view close-ups of faces morphing chronologically in age in 30-second timelapse videos. When asked to tell the age of the face at the very end of the video, the participants almost consistently reported the age of the face that was presented 15 seconds before.

Credit: Mauro Manassi

As we watch the video, we are continuously biased towards the past and so the brain constantly sends us back to the previous ten to 15 seconds (where the face was younger). Instead of seeing the latest image in real time, humans actually see earlier versions because our brain’s refresh time is about 15 seconds. So this illusion demonstrates that visual smoothing over time can help stabilise perception.

What the brain is essentially doing is procrastinating. It’s too much work to constantly deal with every single snapshot it receives, so the brain sticks to the past because the past is a good predictor of the present. Basically we recycle information from the past because it’s more efficient, faster and less work.

This idea – which is also supported by other results – of mechanisms within the brain that continuously bias our visual perception towards our past visual experience is known as continuity fields. Our visual system sometimes sacrifices accuracy for the sake of a smooth visual experience of the world around us. This can explain why, for example, when watching a film we don’t notice subtle changes that occur over time, such as the difference between actors and their stunt doubles.

Repercussions

There are positive and negative implications to our brain operating with this slight lag when processing our visual world. The delay is great for preventing us from feeling bombarded by visual input every day, but it can also risk life-or-death consequences when absolute precision is needed.

This shows a woman's head
To get an idea of the “noisiness” of this visual input, place a phone in front of your eyes and record a live video while you are walking around and looking at different things. Image is in the public domain

For example, radiologists examine hundreds of images in batches, seeing several related images one after the other. When looking at an X-ray, clinicians are typically asked to identify any abnormalities and then classify them. During this visual search and recognition task, researchers have found that radiologists’ decisions were based not only on the present image, but also on images they had previously seen, which could have grave consequences for patients.

Our visual system’s sluggishness to update can make us blind to immediate changes because it grabs on to our first impression and pulls us toward the past. Ultimately, though, continuity fields promote our experience of a stable world. At the same time, it’s important to remember that the judgements we make every day are not totally based on the present, but strongly depend on what we have seen in the past.

Funding:

Mauro Manassi receives funding from the Swiss National Science Foundation fellowship and Carnegie Trust for the Universities of Scotland.

David Whitney receives funding from the National Institutes of Health (US).

About this visual neuroscience research news

Author: Mauro Manassi and David Whitney
Source: The Conversation
Contact: Mauro Manassi and David Whitney – The Conversation
Image: The image is in the public domain

Join our Newsletter
I agree to have my personal information transferred to AWeber for Neuroscience Newsletter ( more information )
Sign up to receive our recent neuroscience headlines and summaries sent to your email once a day, totally free.
We hate spam and only use your email to contact you about newsletters. You can cancel your subscription any time.
  1. How can this finding square with life or death incidents being decided by hair-trigger reflexes, say of a person seeing an absent-minded person about to step off a curb into speeding traffic and reach out to pull them back onto the sidewalk with inches to spare? There would have to be some “manual over-ride” that allows the brain to prioritize the processing of certain categories of input. That, of course, suggests yet another process that PRE-qualifies and sorts the input before it’s placed in the proper physical place at just the right time in the sequence.

    In fact, a subroutine like that would have to occur somewhere at the automatic level of the brain’s control of breathing of the heartbeat, yet be able to access and cross-reference new input against a massive archive of categorized memory fragments — not entire memories, but units of memory analogous to morphemes in language: the smallest unit of speech that possesses meaning.

    Enchanting door you’ve cracked open. I think you haven’t quite thrown it open wide just yet. Keep going!

  2. Particularly intriguing for autistic people like me. If this is done at brain level, it would be reasonable to hypothesise the same happening for smells, touch and audio. I wonder if the autistic brain has a deficit in this ability, leading us to perceive everything in real time, in turn leading to our trademark sensory overload and meltdowns. I suspect it does.

  3. I may be misunderstanding but how would this work with people who play sports? How would a batter be able hit a fastball if his last update was even a few seconds late?

  4. The aging film only works because changes are subtle. Have the image morph into a dog, as an extreme, and you won’t be missing that change. Or have the hair turn grey. Of course, I do believe the brain is “smoothing” things out.

    1. I was thinking the same thing. From my own experience with athletics, I believe the brain has an adrenaline-activated mode of perception, in which things seem to happen in slow motion.

Comments are closed.