Color Naming Models for Color Selection Image Editing

Color Naming Models for Color Selection, Image Editing & Palette Design Jeffrey Heer Stanford University Maureen Stone Tableau Software

“blue”

“green”

“purple”

“salmon” ?

Berlin & Kay (1969) What color is this?

Berlin & Kay (1969)

How might we meaningfully mimic the link between perceived colors and linguistic names? How might such a model enable enhanced visual interfaces?

Contributions A method for building a probabilistic model of color naming from unconstrained judgments. Novel applications enabled by our model that help users express & analyze color in new ways: A color dictionary and thesaurus Name-based pixel selection for image editing Color palette analysis tools

Color Name Modeling 1 Data Collection 2 Data Processing 3 Probabilistic Modeling

Color Name Modeling 1 Data Collection 2 Data Processing 3 Probabilistic Modeling

XKCD http: //xkcd. com/552/

The XKCD Color Survey Ask people to name color swatches 152, 401 sessions (68% male, 27% female, 5% ? ) Median 18 responses / session (IQR: 1030) Perform spam filtering (see paper) Resulting data: 3, 252, 134 color name pairs 2, 956, 183 unique RGB triples

Color Name Modeling 1 Data Collection 2 Data Processing 3 Probabilistic Modeling

Data Processing Goal: construct a color-term count matrix (T) with a reasonable number of colors and terms. 1 Bin colors (5 x 5 x 5 unit bins) in LAB color space

CIE LAB Color Space A perceptual color space based on Opponent-Process Theory 100 L Luminance A Green-Red B Blue-Yellow -100 Axes scaled such that Euclidean distance perceptual distance 0

Data Processing Goal: construct a color-term count matrix (T) with a reasonable number of colors and terms. 1 Bin colors (5 x 5 x 5 unit bins) in LAB color space 2, 956, 183 colors 8, 326 colors

Data Processing Goal: construct a color-term count matrix (T) with a reasonable number of colors and terms. 1 Bin colors (5 x 5 x 5 unit bins) in LAB color space 2 Reduce color term dimensionality (a) Strip all non-alphabetic characters blue-green, blue/green, blue green bluegreen Reduces 132, 259 color terms to 114, 860

Data Processing Goal: construct a color-term count matrix (T) with a reasonable number of colors and terms. 1 Bin colors (5 x 5 x 5 unit bins) in LAB color space 2 Reduce color term dimensionality (a) Strip all non-alphabetic characters (b) Perform dimensional scaling

Color-Term Count Matrix (T) butter aqua yello w green blue red yello w 124 0 0 2 0 32 31 0 0 0 64 0 0 126 0 0 0 119 0 0 0 7 0 0 0 126 0 0 0 43 0 83 0 0 0 6, 889 0 4, 149 6 36 Sum. S 0 0 0 120 16, 400 30, 998 17, 725 14, 400 q || T || = ( ij Tij 2)= 90, 597 pink

Color-Term Count Matrix (T’) butter aqua yello w green blue red yello w 124 0 0 2 0 32 31 0 0 0 64 0 0 126 0 0 0 119 0 0 0 7 0 0 0 126 0 0 0 43 0 83 0 0 0 6, 889 0 4, 149 6 0 Sum. S 0 0 0 120 16, 400 30, 998 17, 725 14, 400 q || T || = ( ij Tij 2) = 90, 597 pink

Color-Term Count Matrix (T’) butter aqua yello w green blue red yello w 124 0 0 2 0 32 31 0 0 0 64 0 0 126 0 0 0 119 0 0 0 7 0 0 0 126 0 0 0 43 0 83 0 0 0 6, 889 0 4, 149 6 0 Sum. S 0 0 0 120 16, 400 30, 998 17, 725 14, 400 q pink || T’ || = ( ij T’ij 2) = 90, 561 f = ||T’ || / ||T || = 0. 99

Data Processing Goal: construct a color-term count matrix (T) with a reasonable number of colors and terms. 1 Bin colors (5 x 5 x 5 unit bins) in LAB color space 2 Reduce color term dimensionality (a) Strip all non-alphabetic characters (b) Perform dimensional scaling f = 0. 99 114, 860 to 66, 526 color terms f = 0. 95 114, 860 to 179 color terms

Data Processing Goal: construct a color-term count matrix (T) with a reasonable number of colors and terms. 1 Bin colors (5 x 5 x 5 unit bins) in LAB color space 2 Reduce color term dimensionality (a) Strip all non-alphabetic characters (b) Perform dimensional scaling (c) Manually correct spelling mistakes

Data Processing Goal: construct a color-term count matrix (T) with a reasonable number of colors and terms. 1 Bin colors (5 x 5 x 5 unit bins) in LAB color space 2 Reduce color term dimensionality (a) Strip all non-alphabetic characters (b) Perform dimensional scaling (c) Manually correct spelling mistakes

Color Name Modeling 1 Data Collection 2 Data Processing 3 Probabilistic Modeling

Probabilistic Naming Model likelihoods of colors and names as multinomial probability distributions. Tabulate conditional probabilities over random variables C (colors) and W (words). p(c|w) = Tc, w / c Tc, w given name Prob of color p(w|c) = Tc, w / w Tc, w Prob of name given color p(W|w) = p(W|c)p(c|w) Association of

Color Saliency The degree to which a color is uniquely named. Measure in terms of the entropy (H) of the conditional probability distribution p(W|c). Saliency(c) = -H(p(W|c)) = w p(w|c) log p(w|c) If everyone agrees on the name, there is no randomness and so the entropy is zero. If people disagree, the number of bits

Color Saliency(c) = -H(p(W|c)) = w p(w|c) log p(w|c)

Orange to Red Low saliency teal, turquoise, green-blue, etc L* = 55

Color Name Distance We can also measure the distance between two colors based on how similarly they are named. A number of distance metrics are feasible. For simplicity we use cosine distance: D(a, b) = 1 – cos(p(W | a), p(W | b)) D(a, b) = 1 – (Ta ¦ Tb / ||Ta|| ||Tb|| )

Applications 1 Color Dictionary & Thesaurus 2 Name-Based Pixel Selection 3 Color Palette Analyzer

Applications 1 Color Dictionary & Thesaurus 2 Name-Based Pixel Selection 3 Color Palette Analyzer

Maximize p(query | name) Most probable colors given by p(c | query) Minimize p(query | name) Maximize LAB distance

Applications 1 Color Dictionary & Thesaurus 2 Name-Based Pixel Selection 3 Color Palette Analyzer

Name-Based Pixel Selection Select an image region… to list probable color names ] Choose a color name… to select all matching pixels Original “Burnt. Orange” “Sky. Blue” [ Ec p(W|c) [ p(C|w) t ] “Olive”

Name-Based Magic

Applications 1 Color Dictionary & Thesaurus 2 Name-Based Pixel Selection 3 Color Palette Analyzer

Color Palette Analyzer Nameable colors facilitate communication It is easier to refer to graphical elements when one can name them unambiguously. Nameable colors are better remembered Experiments (e. g. , Roberson 2000) show improved recall for salient colors. Design goal: increase color name saliency, minimize color name overlap.

Categorical Color Palettes …

Categorical Color Palettes

Categorical Color Palettes

Diverging Color Palettes

Diverging Color Palettes

Diverging Color Palettes

Limitations and Future Work Color Spaces vs. Color Appearance Models CIE LAB vs. CIECAM 02? Data Set Limitations All samples viewed against a white background Refinement & Evaluation of Applications Additional techniques; User studies Incorporation into existing workflows

Color Naming Models for Color Selection, Image Editing & Palette Design Jeffrey Heer Stanford University Maureen Stone Tableau Software vis. stanford. edu/color-names (Apps, Data & Code)

Tools for Colorists Color Harmonization Meier Adobe Kuler Color Brewer PRAVDA Color

Categorical Color Palettes

Diverging Color Palettes

p(C|w) – prob of color given term yello Sum green blue red pink aqua 124 0 0 2 32 31 0 0 0 64 0 126 0 0 0 119 0 0 0 7 0 0 126 0 0 0 43 0 83 0 0 156 0 276 0 169 120 0 83 0 73 w

p(C|w) – prob of color given term yello Sum green blue red pink aqua . 795 0 0 . 027 . 205 . 112 0 0 0 . 877 0 . 457 0 0 0 . 431 0 0 0 . 096 0 0 . 746 0 0 0 . 254 0 1. 0 0 1. 0 w

p(W|c) – prob of term given color yello Sum green blue red pink aqua 124 0 0 2 32 31 0 0 0 64 0 126 0 0 0 119 0 0 0 7 0 0 126 0 0 0 43 0 83 0 0 120 0 0 w 126 127 126 126 120

p(W|c) – prob of term given color yello Sum green blue red pink aqua . 984 0 0 . 016 . 252 . 244 0 0 0 . 504 0 1. 0 0 0 . 944 0 0 0 . 056 0 0 1. 0 0 0 . 341 0 . 659 0 0 1. 0 0 0 w 1. 0 1. 0