3.0 VISUAL PERCEPTION AND COLOR🔗⧉

The previous part, Fundamentals of imaging, followed light from a source through a lens and onto a sensor, and treated the resulting image as a physical measurement. This part asks what that measurement means to an observer. The central idea is that color and vision are computed in the observer, not read off the world: a spectrum of light carries no color until an eye projects it onto a handful of receptors and a brain re-codes and interprets the result. That single fact grounds a surprising amount of imaging engineering, from white balance and color management to JPEG's chroma subsampling and tone mapping.

We start with the hardware. The eye is an optical instrument, the biological twin of the camera built in the last part, so the first chapter walks its optics and its receptor mosaic, and how the eye's own focus errors are measured. From there we build color from the three cones up: why three overlapping pigments make us trichromatic, why an infinite-dimensional spectrum collapses to three numbers, and the metamerism, color blindness, and opponent recoding that follow. The engineering payoff comes next: measuring color with the CIE system, encoding it (linear, gamma, log, and color spaces), and reproducing it across devices. We then turn to how a camera senses color at all, since a bare photosite is colorblind: the Bayer mosaic and its demosaicking, the 3-CCD prism, and the stacked and dispersive sensors that trade hardware for a scene axis. The last two chapters widen the lens: what the visual system does with light once it has it (adaptation, constancy, contrast, and the spatial and temporal sensitivity that every codec exploits), and finally how radically different other animals' eyes and color senses can be, a reminder that human vision is one solution among many.

• Anatomy of the eye and image formation — the eye as an optical instrument.
• Perceptual color and trichromatic vision — three cones, and color as a projection.
• Measuring and encoding color — CIE, color spaces, encoding, and reproduction.
• Sensing color: multiplexing strategies — Bayer, 3-CCD, stacked, and dispersive color sensing.
• Visual processing — adaptation, constancy, contrast, spatial and temporal vision, attention.
• Animal eyes — how other eyes form images and sense color.

Contents of this part

• 3.1 Anatomy of the eye and image formation
• 3.2 Perceptual color and trichromatic vision
  • 3.2.1 Cone Photoreceptors and Perceptual Color
  • 3.2.2 Metamers
  • 3.2.3 Color blindness
  • 3.2.4 Opponent process and the multistage model
  • 3.2.5 So what are the primary colors?
  • 3.2.6 Color as linear algebra
• 3.3 Measuring and encoding color
  • 3.3.1 analysis vs synthesis and non-orthogonality
  • 3.3.2 Measuring color
  • 3.3.3 Chromaticity diagram
  • 3.3.4 Linear vs Gamma vs. log encoding
  • 3.3.5 RGB color spaces
  • 3.3.6 Non-linear perceptually-uniform color space
  • 3.3.7 HSV, HSL, and cylindrical color spaces
  • 3.3.8 Reproducing color
  • 3.3.9 Skin tones
• 3.4 Sensing color: multiplexing strategies
  • 3.4.1 Multiplexing in time
  • 3.4.2 Multiplexing in space: the color filter array
  • 3.4.3 Multiplexing across separate sensors: the beam-splitter
  • 3.4.4 Multiplexing per pixel by dispersion: color routers and nano-prisms
  • 3.4.5 Multiplexing per pixel by tunable absorbers: quantum dots
  • 3.4.6 Multiplexing in depth: stacked photodiodes
  • 3.4.7 Hybrid strategies
  • 3.4.8 Beyond trichromatic capture: full spectrum and multispectral
  • 3.4.9 Spectral sensitivity: matching the eye, catching photons, and white balance
• 3.5 Visual processing
  • 3.5.1 Light adaptation
  • 3.5.2 Lightness constancy
  • 3.5.3 Color constancy
  • 3.5.4 Contrast
  • 3.5.5 Spatial vision
  • 3.5.6 Temporal vision
  • 3.5.7 Attention and eye movements
  • 3.5.8 The visual system makes things up
• 3.6 Animal eyes
  • 3.6.1 The optics: many ways to form an image
  • 3.6.2 The pupil: aperture shape as a clue to ecology
  • 3.6.3 Amphibious vision: focusing in air and in water
  • 3.6.4 The sensor: a retina that is not uniform
  • 3.6.5 Seeing in the dark: sensitivity and its tricks
  • 3.6.6 Acuity: the finest detail, and how to sharpen it
  • 3.6.7 Color vision: opsins remixed
  • 3.6.8 Polarization: the axis we are blind to
  • 3.6.9 Temporal vision: flicker, and motion first

▸ full collapsible outline of this part