Summary: While only 1 in 25 people has synesthesia, a new study reports intuitions about ‘sound colors’ are shared by a greater percentage of people. Sound color perception is mainly driven by the vowels in language.
Source: Radboud University
Does [a:] as in baa sound more green or more red? And is [i:] as in beet light or dark in colour? Even though we perceive speech and color are perceived with different sensory organs, nearly everyone has an idea about what colours and vowels fit with each other. And a large number of us have a particular system for doing so. This is shown in research by linguists from Radboud University and the University of Edinburgh on similarities in the vowel-colur associations perceived by over 1,000 people.
For the writer Vladimir Nabokov, “aa” was the colour of polished ebony and “ee” was yellow. Nabokov had synesthesia: his sensory perceptions mingled with one another. In his case, he saw colors when hearing certain vowels, but many forms of synesthesia are possible. Only 1 in 25 people have synesthesia, but this new research shows that certain intuitions about “sound colors” shared by many more people than this.
“Aa” is more red than green
In this study, over 1,000 people took part in an online test where they chose colors for 16 spoken vowels. A large majority felt that “aa” was more red than green, and “ee” more light than dark, whether they had synesthesia or not.
According to Mark Dingemanse, one of the researchers, “There seems to be a logic to how we link sound and colour, and the structure of language has an important role in this process.”
Sixteen vowels sounds like a lot, but it works like this. When you say “aa”, then move to “oo” as in boot and then to “ee” as in beet, Dingemanse explains, you have visited the three outer points of what linguists call the vowel space. The 16 spoken sounds in our study were evenly distributed over this space.
Vowel system dictates colour associations
Earlier studies have found that color associations are linked to the pitch of the sounds: the higher the pitch, the lighter the color. But the new study shows that colour associations are driven to a greater degree by the vowel system of a language. For example, many participants described sounds that were close to the Dutch vowel “ee” as light green, while nearby sounds resembling “ay” as in say were assigned a different color. The associations are shaped according to how our language carves up the vowel space.
Dingemanse says, “If color associations were purely dependent on acoustical factors, the colors would neatly run into one another like in a rainbow. Instead, we see that sounds are grouped according to the way that our language carves up the vowel space: a few blue spots and then suddenly a red one, with no transition of blue-purple-red. You could say that the vowels have to pass through the sorting machine that is our language before we can link colors to them, even in synesthetes, for whom associations like these are involuntary.”
The researchers used a new method to dig deeper into the structure of the color associations. For each participant, they compared the chosen vowel-color associations with a random sample of 10,000 random associations. They used this to measure how systematic the chosen associations were.
“Synesthetes’ associations were more systematic than those of non-synesthetes,” says Christine Cuskley of Edinburgh University. “But some patterns occur everywhere: people seem to align the vowel space and color space with each other and connect the dots from one space to the other.” For instance, colors chosen for “ee” and “ay” tend to be quite close to each other, while those for “aa” and “oo” are further apart. Automatic associations like those of synesthetes therefore rely on some of the same principles that non-synesthetes use to link vowels and colors.
The study took place as part of the so-called Great National Research Project (GNO), a collaborative venture of Radboud University, the Netherlands Organisation for Scientific Research (NWO) and NTR Broadcasting. In order to better understand the way in which our senses work together, it is necessary to examine a large number of subjects who represent all degrees of synesthesia, from less to more. This was made possible by the GNO, which was set up by Dingemanse and neurobiologist Tessa van Leeuwen of the Donders Institute in Nijmegen. With the support of the NWO, they developed software for performing quick online tests. The researchers made the software publicly available so that others could continue to work with it. Future publications can thus share additional new insights into how synesthesia works and how language influences our perception.
Mark Dingemanse – Radboud University
The image is credited to Mark Dingemanse Radboud University.
Original Research: Open access
“Cross-modal associations and synesthesia: Categorical perception and structure in vowel–color mappings in a large online sample”
Cuskley, C., Dingemanse, M., Kirby, S. et al.Behavioral Research (2019) doi:10.3758/s13428-019-01203-7
Cross-modal associations and synesthesia: Categorical perception and structure in vowel–color mappings in a large online sample
We report associations between vowel sounds, graphemes, and colors collected online from over 1,000 Dutch speakers. We also provide open materials, including a Python implementation of the structure measure and code for a single-page web application to run simple cross-modal tasks. We also provide a full dataset of color–vowel associations from 1,164 participants, including over 200 synesthetes identified using consistency measures. Our analysis reveals salient patterns in the cross-modal associations and introduces a novel measure of isomorphism in cross-modal mappings. We found that, while the acoustic features of vowels significantly predict certain mappings (replicating prior work), both vowel phoneme category and grapheme category are even better predictors of color choice. Phoneme category is the best predictor of color choice overall, pointing to the importance of phonological representations in addition to acoustic cues. Generally, high/front vowels are lighter, more green, and more yellow than low/back vowels. Synesthetes respond more strongly on some dimensions, choosing lighter and more yellow colors for high and mid front vowels than do nonsynesthetes. We also present a novel measure of cross-modal mappings adapted from ecology, which uses a simulated distribution of mappings to measure the extent to which participants’ actual mappings are structured isomorphically across modalities. Synesthetes have mappings that tend to be more structured than nonsynesthetes’, and more consistent color choices across trials correlate with higher structure scores. Nevertheless, the large majority (~ 70%) of participants produce structured mappings, indicating that the capacity to make isomorphically structured mappings across distinct modalities is shared to a large extent, even if the exact nature of the mappings varies across individuals. Overall, this novel structure measure suggests a distribution of structured cross-modal association in the population, with synesthetes at one extreme and participants with unstructured associations at the other.