Summary: Researchers have successfully reproduced illusory motion in deep neural networks that were trained for perception. The DNNs were able to accurately reproduce the direction of illusory rotation humans observe while looking at a popular optical illusion.
Source: National Institutes of Natural Sciences.
Deep neural networks (DNNs), which have been developed with reference to the network structures and the operational algorithms of the brain, have achieved notable success in a broad range of fields, including computer vision, in which they have produced results comparable to, and in some cases superior to, human experts. In recent years, DNNs have also been expected to be useful as a tool for studies of the brain.
A research team led by associate professor Eiji Watanabe of the National Institute for Basic Biology successfully reproduced illusory motion by DNNs trained for prediction.
The DNNs are based on predictive coding theory, which assumes that the internal models of the brain predict the visual world at all times and that errors between the prediction and the actual sensory input further refine the internal models. If the theory substantially reproduces the visual information processing of the brain, then the DNNs can be expected to represent the human visual perception of motion.
Optical flow vectors detected between a pair of consecutive predictive images of the illusion. Red bars denote the direction and magnitude of vectors, yellow dots denote the start points of the vectors. The left is a single ring of the rotating snake illusion, and the right is a negative control image. Credit: NIBB.
In this research, the DNNs were trained with natural scene videos of motion from the point of view of the viewer, and the motion prediction ability of the obtained computer model was verified using a rotating propeller in unlearned videos and the “Rotating Snake Illusion”. The computer model accurately predicted the magnitude and direction of motion of the rotating propeller in the unlearned videos. Surprisingly, it also represented the rotational motion for illusion images that were not moving physically, much like human visual perception. While the trained network accurately reproduced the direction of illusory rotation, it did not detect motion components in negative control pictures wherein people do not perceive illusory motion.
People perceive clockwise or counter-clockwise motion depending on color alignment. Negative controls (non-illusions) for which people perceive no motion are presented in the right panel. NeuroscienceNews.com image is credited to Akiyoshi Kitaoka.
Dr. Watanabe said, “This research supports the exciting idea that the mechanism assumed by the predictive coding theory is a basis of motion illusion generation. Using sensory illusions as indicators of human perception, deep neural networks are expected to contribute significantly to the development of brain research.”
Funding: This research was conducted as a collaborative research project by the National Institute for Basic Biology, SOKENDAI (the Graduate University for Advanced Studies), Ritsumeikan University, the National Institute for Physiological Sciences, and Sakura Research Office.
Source: National Institutes of Natural Sciences
Publisher: Organized by NeuroscienceNews.com.
Image Source: NeuroscienceNews.com images are credited to NIBB and Akiyoshi Kitaoka.
Original Research: Open access research in Frontiers in Psychology.
doi:10.3389/fpsyg.2018.00345
[cbtabs][cbtab title=”MLA”]National Institutes of Natural Sciences “Illusory Motion Reproduced by Deep Neural Networks Trained for Perception.” NeuroscienceNews. NeuroscienceNews, 20 March 2018.
<https://neurosciencenews.com/ai-optical-illusion-8671/>.[/cbtab][cbtab title=”APA”]National Institutes of Natural Sciences (2018, March 20). Illusory Motion Reproduced by Deep Neural Networks Trained for Perception. NeuroscienceNews. Retrieved March 20, 2018 from https://neurosciencenews.com/ai-optical-illusion-8671/[/cbtab][cbtab title=”Chicago”]National Institutes of Natural Sciences “Illusory Motion Reproduced by Deep Neural Networks Trained for Perception.” https://neurosciencenews.com/ai-optical-illusion-8671/ (accessed March 20, 2018).[/cbtab][/cbtabs]
Abstract
Illusory Motion Reproduced by Deep Neural Networks Trained for Prediction
The cerebral cortex predicts visual motion to adapt human behavior to surrounding objects moving in real time. Although the underlying mechanisms are still unknown, predictive coding is one of the leading theories. Predictive coding assumes that the brain’s internal models (which are acquired through learning) predict the visual world at all times and that errors between the prediction and the actual sensory input further refine the internal models. In the past year, deep neural networks based on predictive coding were reported for a video prediction machine called PredNet. If the theory substantially reproduces the visual information processing of the cerebral cortex, then PredNet can be expected to represent the human visual perception of motion. In this study, PredNet was trained with natural scene videos of the self-motion of the viewer, and the motion prediction ability of the obtained computer model was verified using unlearned videos. We found that the computer model accurately predicted the magnitude and direction of motion of a rotating propeller in unlearned videos. Surprisingly, it also represented the rotational motion for illusion images that were not moving physically, much like human visual perception. While the trained network accurately reproduced the direction of illusory rotation, it did not detect motion components in negative control pictures wherein people do not perceive illusory motion. This research supports the exciting idea that the mechanism assumed by the predictive coding theory is one of basis of motion illusion generation. Using sensory illusions as indicators of human perception, deep neural networks are expected to contribute significantly to the development of brain research.
