2.2 Perceptual color and trichromatic vision🔗⧉

Color has several meanings from physical to different layers of perception and all these stages matter to photography and can be the cause of much confusion. This chapter follows that whole pipeline in order: from physical color (a spectrum), to three cone responses, to the opponent recoding, up to the high-level assessments where illuminant discounting, color constancy, color names, and focal prototypes live. Each stage is a different, equally valid answer to "what color is this." The rest of human vision — light adaptation, lightness and color constancy, contrast, spatial and temporal sensitivity, and attention — is taken up in the next chapter, Human Vision.

One point is worth flagging: visual perception is measurable. Psychophysics, the experimental study of the map from stimulus to sensation, is quantitative and reproducible. Color-matching, hue-cancellation, and contrast-sensitivity experiments return tight numbers that repeat across observers and across laboratories, solid enough to be frozen into standards such as the CIE standard observer and the perceptual models inside every camera and codec.

2.2.1 Visual system anatomy:🔗⧉

Light entering the eye passes through the cornea and the lens, the eye's two refracting elements, the same Snell's-law optics as a camera lens (the cornea does most of the bending; the lens fine-tunes focus by changing shape, accommodation). The pupil, the iris's adjustable opening, is the eye's aperture. The image lands on the retina, a thin sheet of neural tissue lining the back of the eyeball (Figure 2.2.1).

The retina's light-sensitive cells come in two families. Rods are exquisitely sensitive and handle dim, night (scotopic) vision, but there is only one kind, so rod vision is colorless. Cones are less sensitive, work in daylight (photopic vision), and come in three types — the basis of color. Their distribution across the retina is wildly non-uniform (Figure 2.2.2). Cones are packed densely in the fovea, the tiny central pit we point at whatever we are scrutinizing; rods dominate the periphery. This is why you cannot read text out of the corner of your eye, and why faint stars are easier to see slightly off-center.

Physically these are minute cells. In the fovea the cones are the finest of all, only about $2\,\mu\text{m}$ across, packed at a center-to-center spacing near $2.5\,\mu\text{m}$ that sets the eye's acuity limit; toward the periphery cones fatten to $5$–$8\,\mu\text{m}$. Rods are slimmer still, roughly $1$–$2\,\mu\text{m}$ in diameter. A few micrometers is therefore the biological yardstick against which a camera's photosite is measured, and modern phone pixels have shrunk to below the size of a foveal cone (→ photosite sizes).

Two quirks of this layout matter. First, there are essentially no short-wavelength (S, "blue") cones in the very center of the fovea, and far fewer S cones than long- and medium-wavelength (L and M) cones overall. As a result blue is sensed at low resolution and is even slightly out of focus — the eye's chromatic aberration puts blue at a different focal plane — yet we perceive a crisp blue everywhere, because the brain fills it in. Second, luminance (our sense of brightness) is roughly the sum of the L and M cone responses, so green carries most of our spatial acuity. This single fact echoes through the whole book: it is why camera color-filter mosaics (the Bayer pattern) use twice as many green cells as red or blue, and why image compression can throw away color detail but not brightness detail.

The retina is not a passive film: it does substantial processing before anything leaves the eye. The photoreceptors do not report straight to the brain; they feed layers of intermediate cells that in turn drive the retina's output neurons, the ganglion cells, whose long fibers bundle together into the optic nerve, the cable to the brain. Already here the signal is reorganized: each output cell no longer reports raw brightness but compares a small spot to the ring of retina around it, a center-surround response that reacts to local contrast and edges rather than absolute level (the seed of the edge enhancement we return to under Spatial vision). From the eye the pathway passes through a relay station deep in the brain (the lateral geniculate nucleus) and on to the primary visual cortex at the back of the head, then to higher areas that handle motion, form, and recognition. The lesson for us: much of what we naïvely attribute to "the image" is really a construction, assembled in stages from a heavily pre-processed retinal signal.

2.2.2 Color🔗⧉

Even though cones are known to be the color vision receptors, a single cone does not measure wavelength — it produces one number, and stimuli of different wavelengths can yield the same response.

A given cone type $k$ has a spectral sensitivity $c_k(\lambda)$: a curve saying how strongly it responds to light of each wavelength (Figure 2.2.4).

When light with spectrum $E(\lambda)$ arrives, the cone reports a single response, the spectrum weighted by its sensitivity and summed over all wavelengths (Figure 2.2.5):

Mathematically this is a dot product. In the case of monochromatic light (a single wavelength $\lambda$) it reduces to the scalar product of the cone's sensitivity at $\lambda$ and the light's intensity for $\lambda$.

As Figure 2.2.6 shows, lights at different wavelengths can yield the same response if their intensity ratio is the inverse of the cone's spectral response ratio for those two wavelengths. A cone conflates color with intensity.

While a single cone cannot distinguish wavelength, the combination of three cone responses provides much more information (click on "show other cones" in Figure 2.2.6 ). This is the trichromatic theory of color vision, pioneered by Thomas Young and put on a quantitative footing by Hermann von Helmholtz. Three cone types give three numbers $(r_L, r_M, r_S)$, and that triple is all the information the eye keeps about the spectrum. An infinity of wavelengths collapses to three.

It helps to make the projection concrete as linear algebra. Discretize the spectrum into $N$ wavelength bins and stack the samples into a vector $\mathbf{E} \in \mathbb{R}^N$. Stack the three sensitivity curves as the rows of a $3 \times N$ matrix $\mathbf{C}$, one row per cone type. Then the three cone responses are just a matrix–vector product,

(Figure 2.2.7). In reality wavelength is continuous, so $N \to \infty$ and the sum becomes the integral above — color sensing is genuinely an infinite-dimensional input squeezed through a three-dimensional bottleneck.

We established above that a single cone cannot distinguish wavelengths. Only by comparing the three cones' outputs can the brain begin to separate the two. However, this ability is limited. This is unsurprising because trichromatic vision summarizes a while spectrum with only three numbers.

Because the map $\mathbf{E} \mapsto \mathbf{r}$ is a projection from infinite dimensions to three, it has an enormous null space: vastly many different spectra produce the same triple $(r_L, r_M, r_S)$ and therefore look exactly the same. Two such spectra are metamers. Metamerism is not a defect — it is what makes color reproduction possible at all. A screen with only three primaries can match (almost) any real spectrum, not by reproducing it, but by producing a metamer of it. The whole edifice of color technology in the next chapter rests on this.

Metamerism also has an important limitation, and it follows from the very first lesson of this part. The light that reaches the eye is the product of a surface's reflectance and the illuminant (→ see Big lesson L2.6, illumination × reflectance), so a metameric match is really a match between two products. Change the illuminant — daylight for shop fluorescent, sunlight for tungsten — and you multiply both spectra by a different shape, and two reflectances that matched under one light can visibly diverge under another (Figure 2.2.11). The damage is worst under spiky illuminants (fluorescent tubes, white LEDs, sodium lamps), whose spectra are a few narrow peaks rather than a smooth curve. A spiky light samples the two reflectances only where its peaks fall, so small differences that a smooth daylight would average over get seized on and amplified, and the match falls apart. This is one of the situations where spiky illumination spectra are bad; we meet others under white balance and color rendering. The effect is easy to see for real (Figure 2.2.10): two gray patches printed as a metameric pair in Billmeyer & Saltzman's Principles of Color Technology look identical under daylight but split apart, one turning pink, under a warm halogen lamp. This illuminant metamerism is a real and expensive problem: a printer's inks are mixed to be a metamer of the target colors under the proofing light, then drift apart under the customer's living-room bulbs; and two pieces of an outfit, dyed to match in the shop, can clash in daylight. It is why color-critical work always specifies the viewing illuminant — "matches here" is never "matches everywhere."

In summary, going from physical color to the cone color is simple linear algebra. A linear projection or a big Nx3 matrix if you discretize wavelength. There is loss of information like with any projection, but the mathematical model is simple linear algebra. The part that makes the linear algebra messyare twofold. First, The three cone "axes" (the spectral responses) are not orthogonal — look again at Figure 2.2.4, where L and M overlap almost completely. Because $\mathbf c_L$ and $\mathbf c_M$ sit almost on top of each other across nearly the whole spectrum, their dot product $\mathbf c_L\!\cdot\mathbf c_M$ is large and positive: the two axes lean hard toward one another. In practice this means cross-talk — there is essentially no physical light that stimulates M without also stimulating L and S. Shine any single wavelength and at least two cone types respond together; the M signal can never be moved on its own — the only way to move it alone is to abandon light and stimulate the cones directly, cone by cone (the Oz sidebar above, and its extreme form in Retinal projection). Second, spectra, being physical power densities, cannot be negative, so the cone responses as well as any light we may use to reproduce color may not be negative. Both facts conspire against the comfortable situation in linear algebra where bases are orthonormal and each axis can be handled independently and where negative numbers are available to span the whole space.

Color blindness is what happens when this projection loses even more information. A "dichromat" is missing one cone type (most commonly through the L/M genes), so the spectrum is projected onto two numbers instead of three; many spectra that look distinct to a trichromat now collapse into the same response — the classic red–green confusion. The standard Ishihara plates (Figure 2.2.13) exploit exactly this, hiding a figure that a trichromat sees and a dichromat cannot. The deeper point is one of perspective: a trichromat is also projecting away an infinity of spectral information. We are all color-blind relative to the full spectrum — and, as the next chapter (Animal eyes) shows, relative to other animals too.

Finally, the brain does not stop at three cone numbers. It re-codes color through a multistage model (simplified here; the full version is in Reinhard et al., Color Imaging):

• Stage 1 — cones. Three heavily overlapping L/M/S responses, the projection above.
• Stage 2 — opponent recoding, in the retina and the lateral geniculate nucleus. The L/M/S signals are re-mixed into three opponent channels: light–dark, red–green, and blue–yellow. This decorrelates the channels (since L and M are so redundant, their difference carries the color information efficiently) and matches the four "unique hues" we feel — the subject of the next section.
• Stage 3 — appearance. A higher-level organization closer to hue, saturation, and brightness (a hue-saturation-value-like, or HSV-like, arrangement), where context, adaptation, and memory shape the final percept — the domain of color-appearance models in the next chapter.

The recoding is not arbitrary. Color names cluster around shared focal colors across unrelated languages (Berlin & Kay's classic study, and the broader World Color Survey) — evidence that the structure is perceptual, built into the wiring rather than the culture. The recoding has support from two directions: psychophysics (Hurvich and Jameson's hue-cancellation experiments) and physiology (De Valois's recordings of opponent cells).

2.2.3 Opponent process and the multistage model🔗⧉

The two-stage story is worth stating cleanly: color is trichromatic at the cones, then opponent in the retina and lateral geniculate nucleus. The three cone signals are recombined into three opponent channels — red–green (roughly $L - M$), blue–yellow (roughly $S - (L+M)$), and light–dark (roughly $L + M$, the luminance channel). In symbols this is just a fixed linear map,

with $\mathbf{M}$ a $3 \times 3$ matrix — the perceptual ancestor of the luma–chroma transforms (the $Y'C_bC_r$ of JPEG) we will use later (Figure 2.2.15).

This model earns its keep in two ways. It explains afterimages: stare at a saturated red patch until the red–green channel adapts, then look at white and the channel rebounds the other way, painting a green ghost (likewise blue→yellow). And it explains why luma/chroma encodings compress so well: the luminance channel ($Y'$, computed on gamma-encoded RGB — see Linear vs Gamma next chapter) carries most of the detail we can resolve, while the two chroma channels can be heavily subsampled with little visible loss — the foundation of JPEG's chroma subsampling, and a direct prediction of the opponent model that we will confirm under Spatial vision.

2.2.4 So what are the primary colors?🔗⧉

Everyone has argued about this. Is it red-green-blue (RGB)? The grade-school red-yellow-blue (RYB)? Cyan-magenta-yellow (CMY)? The perennial muddle has two separate causes, and neither of them is a fact about light.

The first cause is the opponent recoding we just met. Because the visual system re-wires the three cone signals into red–green, blue–yellow, and light–dark, red, green, blue, and yellow all feel "primary" to us perceptually — there are four psychological unique hues, not three. This is why red-yellow-blue feels so natural to painters, and why the cones' actual L/M/S sensitivities match nobody's intuition of "primary" colors: our intuition is reporting the opponent stage, not the cones.

The second cause is additive versus subtractive synthesis, a matter of technology (taken up fully in the next chapter). RGB are the additive primaries — the colors of light you add together on a screen. CMY are the subtractive primaries — the inks or filters that remove light from white. The grade-school RYB is just a folk approximation of subtractive mixing.

So "primary color" silently conflates a perception fact with a technology choice — two different questions with two different answers, neither of which is intrinsic to the physics of light. As the lecture put it, the opponent rewiring "is one of the many causes of all the confusion about what's a primary color." (See the Glossary entry for primary.)