Summary: By pairing virtual reality (VR) eye-tracking headsets with advanced machine learning and Large Language Models (LLMs), the research team discovered that our eyes are driven by deep “conceptual priorities.” Our brains unconsciously hunt for objects that carry personal, abstract meaning to us. This visual signature is so consistent over time that AI can accurately identify an individual simply by analyzing the thematic connections between the items they choose to look at.
Key Facts
- The Concept Priority Blueprint: A conceptual priority is a personal mental bias that dictates what jumps out at us visually. For instance, an American flag and a football look entirely different physically, but an LLM can group them under abstract themes like “patriotism” or “sports culture.” The study proved our eyes inherently spend more time seeking out objects that match our unique internal thematic interests.
- The Three Perceptual Stages of Sight: The researchers discovered that regardless of personal interests, all human visual exploration follows a rigid, three-part biological timeline when entering a new space:
- Stage 1 (0 to 2 Seconds): Spatial Orientation. The brain scans the physical dimensions, such as the room’s horizon and center.
- Stage 2 (2 to 8 Seconds): Visual Salience. Gaze shifts to prominent, physically striking objects and people.
- Stage 3 (8 to 16 Seconds): Interpretive Semantics. The brain transitions into a deep conceptual mode, focusing heavily on what the objects mean.
- The LLM Identity Breakthrough: When analyzing the eye-tracking data, the LLM was able to successfully identify individuals out of the cohort based solely on their gaze patterns. Remarkably, tracking the meaning of the objects a person looked at (via the LLM) was far more accurate at identifying them than tracking the raw physical features of the objects.
- Temporal Stability Across Weeks: When participants returned a week later to view entirely new scenes, the AI models built from their previous week’s data accurately predicted exactly what objects would catch their eye. This proves that individual gaze choices are stable, personality-level traits that persist over time.
- The Commercial Surveillance Threat: The researchers warn that as consumer VR and AR headsets become integrated into daily life, eye-tracking data could become highly intrusive. Advertisers could use this tech to passively harvest your political leanings, hobbies, and psychological profile based simply on what you look at, moving far beyond traditional web clicks.
- Pioneering Clinical Applications for Autism: This study marks the first time an LLM has been used to model human visual gaze. Dr. Amanda Haskins notes this AI framework could revolutionize early autism screenings. By determining whether a child’s visual avoidance of faces is purely a visual processing trait or a deeper conceptual choice, clinicians could diagnose autism closer to age two, lowering the current national average diagnosis age of four and enabling faster educational support.
Source: Dartmouth University
Walk into a crowded coffee shop, and what catches your eye as you take in the scene could say as much about you as the spirals on your fingertips or the mutations in your DNA.
Eye movements are so unique, in fact, that they could be used to identify you through the objects that have personal meaning, according to a new study by Dartmouth researchers in the Proceedings of the National Academy of Sciences.
The findings reveal the depth to which we subjectively evaluate what’s around us, while also suggesting that in a world of constant surveillance, we may be giving away more personal information than we realize.
Psychologists have long studied where people consciously or unconsciously focus their attention as they scan a new environment. While we usually come away with a similar understanding of the place itself, each person has distinct perception of how they got there, where they look, and for how long.
It’s that variation that the study’s senior author, Caroline Robertson, and her team studied.
“From the earliest moments of taking in a new environment, we make radically different choices about what we pay attention to,” says Robertson, who is an associate professor of psychological and brain sciences at Dartmouth. “This work suggests that our latent conceptual priorities are embedded in the signatures of our gaze.”
A conceptual priority is a kind of personal bias that shapes what jumps out at us visually. A flag and a football, for example, look nothing alike physically, but they are connected by abstract ideas of identity such as patriotism or the United States. The study suggests that, in new environments, we spend more time seeking out information that is conceptually rich and personally meaningful to us.
And what people look for in an unfamiliar environment can distinguish one person from another by the objects and concepts that mean something to them, like a personality fingerprint, the researchers report.
Looking at where people look
Robertson studies visual attention and became focused on the individual differences that kept popping up in her experiments. With the study’s first author, Amanda (“AJ”) Haskins, who received her PhD from Dartmouth in 2024, and former research assistant Katherine Packard ’23, Robertson had about 60 study participants wearing VR headsets immerse themselves in a series of images of everyday scenes, including an auto repair shop, a public swimming pool, and an airport. Participants were free to turn their heads, move their bodies, and look where they pleased in the 16 seconds allotted for each image.
Meanwhile, the researchers used eye-tracking data recorded by the headsets to model each individual’s gaze pattern. They created a machine-learning model to recreate where participants looked within each space; a vision model to recreate the objects that held their attention; and a large language model, or LLM, to look at the conceptual themes that tie these objects together.
The LLM, which are the deep-learning systems that power artificial intelligence, generated captions for each image it processed that hinted at potential storylines. For example, one caption read, “a flag that is on the wall and could be signaling national identity,” while another read, “a missile that is for military equipment and could be part of an aeronautics display.”
When the researchers analyzed the results, they found that the vision model and the LLM could identify individuals by their unique eye movements, and that the LLM, which specifically encoded their conceptual preferences, could do this most accurately. The conceptual map, in other words, was most predictive.
Objects that don’t look alike but are thematically linked could be used to tell participants apart when considered as a whole. For example, one person taking in an office scene first looked at writing-related items such as keyboards and notepads, while another individual focused on architectural elements like moldings and decorative backsplashes. These objects’ conceptual similarities reveal each participant’s unique preferences or interests.
These individual preferences were long lasting, the researchers find. When half the group returned a week later to explore a new set of scenes, the models built from their earlier eye-tracking data accurately predicted the visual features that would grab their attention.
“This suggests that individual differences in gaze patterns contain stable, personality-level preferences that extend beyond testing days,” Robertson says.
Though their gaze patterns varied, the participants had three main perceptual stages. In the first two seconds of taking in a new scene, their gaze focused on spatial dimensions, like the image’s horizon and center, before shifting to prominent visual elements, and after about eight seconds, to the meanings encoded in them.
That intuitively made sense to the researchers. We typically orient ourselves in space and then check out objects and people before transitioning to an interpretive mode that attempts to understand what it all means.
The richer the conceptual information that the LLM received, the more fine-grained distinctions it could pick up on. The researchers found that longer captions containing more context, such as “a hat that is on her head and could be keeping the sun from her eyes,” seemed to elicit more distinctive responses than just “a hat that is on her head.” The more distinctive the eye patterns, the easier they were for the LLM to pick up.
What the eyes give away
The researchers point out that eye-tracking data alone may not reveal our politics or personalities. But their findings suggest that VR and AR could be more intrusive than we realize, potentially giving away more personal data to advertisers than we do now with our clicks across the web.
The study may be the first to use an LLM to model visual gaze—and it will likely not be the last. The researchers are hopeful that the novel AI methods used in the study could have clinical applications.
“Individual gaze differences aren’t random, but rather, are consistent from place to place and stable over time,” says Haskins, now a postdoctoral researcher at the University of California, San Diego. “That’s important if we want to use gaze as a clinical marker of conditions like autism.”
One hallmark of autism is a reduced focus on faces, but it’s been unclear whether face avoidance is more visual than conceptual. The approach the researchers used could help to distinguish between the two.
It could also make earlier diagnosis of autism possible. Symptoms can show up as early as two years old, but currently the national average age at diagnosis is four. “The sooner you could know that a child is processing the world differently, the sooner you could augment the teaching environment,” Robertson says.
The team’s next steps include exploring whether multimodal models that track both visual and cognitive attention could improve predictions further. They also want to test whether the conceptual priorities they’ve identified vary systematically across cultures or clinical groups.
Key Questions Answered:
A: It isn’t about a single object; it’s about the abstract themes that connect everything you look at. If you walk into an office, you might spend eight seconds looking at keyboards, fountain pens, and notebooks, while someone else focuses on wall moldings and decorative tiles. A computer vision model just sees shapes, but an LLM understands the underlying story. It recognizes that your eyes consistently hunt for “writing and communication tools,” while the other person prioritized “architectural design.” These semantic maps are so highly individualized and consistent that they act as a visual fingerprint.
A: When you browse the web, advertisers only know what you explicitly click on or search for. But when you wear a Virtual Reality or Augmented Reality headset, built-in eye-tracking cameras record exactly where your eyes land every millisecond, completely outside your conscious control. Because this Dartmouth study proves that your gaze patterns directly expose your subconscious biases, hobbies, and personal priorities, companies could use this tech to passively map out your personality, politics, and vulnerabilities without your explicit consent.
A: One of the classic hallmarks of autism is an avoidance of looking at human faces, but scientists have never known if this behavior is driven by a basic visual discomfort or a deeper conceptual choice. By using this study’s AI triad, researchers can analyze the exact timeline of a child’s gaze to distinguish between the two. Because these visual signatures are incredibly stable and measurable in toddlers as young as two, this approach could allow doctors to diagnose autism two years earlier than the current national average, letting parents customize their child’s learning environment during critical developmental windows.
Editorial Notes:
- This article was edited by a Neuroscience News editor.
- Journal paper reviewed in full.
- Additional context added by our staff.
About this visual neuroscience research news
Author: Morgan Kelly
Source: Dartmouth College
Contact: Morgan Kelly – Dartmouth College
Image: The image is credited to Neuroscience News
Original Research: Open access.
“Conceptual priorities shape individual gaze patterns during naturalistic visual attention” by Amanda J. Haskins, Katherine O. Packard, and Caroline E. Robertson. PNAS
DOI:10.1073/pnas.2604369123
Abstract
Conceptual priorities shape individual gaze patterns during naturalistic visual attention
Our visual landscape consists of not only people, places, and objects (e.g., “soldier,” “stadium,” “flag”) but also the conceptual relationships that unite them (e.g., “patriotism”). Because conceptual knowledge varies across individuals, this level of structure may support individualized patterns of attentional selection during naturalistic scene viewing.
Here, we ask whether individuals’ gaze patterns reflect, in part, latent attentional priorities organized in conceptual space. Participants (N = 61) freely explored a diverse set of immersive real-world scenes (N = 100) in head-mounted VR while their gaze position was continuously recorded.
We modeled gaze behavior using spatial, visual, and conceptual feature spaces, leveraging embeddings from large vision and language models, to uncover the latent priorities guiding individuals’ unique patterns of selective attention across environments. Individuals exhibited stable and idiosyncratic gaze patterns across scenes and test–retest sessions, consistent with trait-like individual differences in attention. Spatial, visual, and conceptual feature spaces each explained unique variance in individual gaze patterns, with conceptual features contributing variance beyond that explained by spatial and visual features alone.
Notably, language-model–based predictions were particularly effective at capturing these individualized patterns. Together, these findings indicate that naturalistic visual attention is structured at multiple levels—including a conceptual level—revealing stable individual differences in how people sample and prioritize information across complex visual environments.
