Experiment
A small study on grapheme-color synesthesia and whether embedding spaces reflect our most common letter–color associations.
↓ scroll
Background
Grapheme-color synesthesia is a perceptual phenomenon in which written letters and numbers automatically evoke specific color experiences. For a synesthete, for instance, A might always appear (or feel like) red. The associations are stable across time and some can be surprisingly consistent across individuals.
The color mosaic above shows each letter in its modal synesthetic color; the color most commonly chosen by synesthetes in a large empirical study. Hover any tile to see the percentage.
This project asks whether embedding models, the mathematical representations of language inside modern AI, have picked up these associations from text. We computed cosine similarity between letter and color embeddings across five model families, then compared the resulting 26 × 11 similarity matrices to empirical data from 6,588 synesthetes collected by Witthoft, Winawer & Eagleman (2015). Each RGB color pick was mapped by Witthoft et al. to one of Berlin & Kay's 11 basic color categories, producing a frequency matrix of how often each letter gets each color.
Interactive
Toggle between the synesthesia frequency data and each embedding condition. Color intensity reflects the value. Red is for frequency, green for Word2Vec, purple for RoBERTa, teal for Voyage, blue for GPT. Hover any cell for details.
Results
Spearman ρ measures overall rank correlation across all 286 letter–color pairs. Hit rate counts how often the model's top-similarity color for each letter matches the most common synesthetic color.
Chance baseline for hit rate: 1/11 ≈ 9% (~2–3 / 26 letters)
Key finding
The three GPT conditions use the same model and the same 26 letters, changing only the wording. Asking for "the letter A" instead of bare "A" shifts the correlation from near-zero to strongly significant. RoBERTa-large shows the same jump (ρ = 0.41 with full phrasing), while Word2Vec is flat regardless.
Per-letter breakdown
For each letter, did the model's top-similarity color match the most common synesthetic color? Cell color = that letter's modal synesthetic color.
Left = GPT "letter A" / "red" | Right = GPT "the letter A" / "the color red"
Methods
Five model families were tested, each in one or more phrasing conditions. For each condition, cosine similarity was computed between every letter embedding and every color embedding, producing a 26 × 11 matrix.
Distributional embeddings trained by predicting surrounding words. Tested on Google News (3B words) and a custom literary corpus (~50 Project Gutenberg novels). Vectors are fixed — context has no effect. Phrasing a query as "letter A" is done by averaging the vectors for "letter" and "A".
Global Vectors for Word Representation, trained on 2B tweets (200-dimensional). The informal, culturally-saturated nature of Twitter text makes this the strongest static baseline, with a small but significant correlation (ρ = 0.16).
all-roberta-large-v1 via Sentence Transformers. A 24-layer BERT-style encoder fine-tuned for semantic similarity. Shows the same phrasing effect as GPT: near-zero with bare letters, ρ = 0.41 with "the letter A" — and uniquely gets A → Red correct.
Voyage AI's general-purpose retrieval model. Shows significant signal with full phrasing (ρ = 0.22, p < 0.001), weaker than GPT or RoBERTa. Bare and "letter A" conditions are flat. The phrasing effect is present but attenuated.
OpenAI's text-embedding-3-small. The strongest overall performer (ρ = 0.43 with "letter A"). Gets 10/26 modal colors right with the full "the letter A" / "the color" phrasing — the highest hit rate — but misses A → Red in all three conditions.