2.0 FUNDAMENTALS of Imaging⧉
A photograph records light that has traveled from a source, interacted with a scene, and reached a sensor or eye. Light leaves a source, such as the sun, a lamp, or a phone screen, and travels until it strikes objects in a scene, scatters according to what those objects are made of, and a sliver of it finally passes through a lens onto a sensor (or, in the eye, onto the retina). The sensor counts photons; the camera turns those counts into a grid of numbers; a visual system, silicon or biological, interprets the result (Figure 2.0.1). Photography means, almost literally, writing with light, from the Greek phōs ("light") and graphē ("drawing").
This part follows that journey, because every later trick in the book depends on it. We begin with light itself: what it is (wave, ray, or photon), how it carries color, and how it behaves when it meets a surface (the first chapter). Then we cross straight into the observer, because color is made there: the eye and visual system, why three cones make us trichromatic, and color as the projection of a whole spectrum onto three numbers, together with its engineering payoff, color technology (measuring color with the CIE system, encoding it in linear / gamma / log, and reproducing it). Only then do we ask how a camera turns the light filling a room into a flat picture: the geometry of image formation, from the pinhole to the lens and the limits geometry imposes, such as perspective, focus, and depth of field, and how a pixel value is an integral of the incoming light, and the sensor and the noise that floor what we can see. Those threads converge in the two abstract chapters that frame the whole book: imaging as a linear system and imaging as an inverse problem. The part closes by stepping back to the medium itself and asking what a picture fundamentally cannot do: its limitations against the eye: flatness, a frozen instant, a finite frame, and limited dynamic range and gamut. The craft of a real camera as an instrument, and the question of why a photograph is never a neutral recording, follow in the next part, Photography.
Contents of this part
- 2.1 Light and physics
- 2.1.1 Light: rays, waves, and the spectrum
- 2.1.2 How light is created
- 2.1.3 Light power and brightness
- 2.1.4 Summing coherent vs. incoherent light
- 2.1.5 Reflection, refraction, and what happens at a surface
- 2.1.6 Polarization
- 2.1.7 The color of objects: illumination times reflectance
- 2.1.8 The BRDF and the look of materials
- 2.1.9 Illuminants
- 2.1.10 Radiometry: radiance, irradiance, exposure, and falloff
- 2.1.11 Global illumination
- 2.1.12 Wave effects, diffraction, and the diffraction limit
- 2.2 Perceptual color and trichromatic vision
- 2.3 Human Vision
- 2.3.1 Light adaptation
- 2.3.2 Lightness constancy
- 2.3.3 Color constancy
- 2.3.4 Contrast
- 2.3.5 Spatial vision
- 2.3.6 Temporal vision
- 2.3.7 Attention and eye movements
- 2.3.8 The visual system makes things up
- 2.4 Animal eyes
- 2.5 Measuring and encoding color
- 2.6 Pinhole image formation and linear perspective
- 2.6.1 Pinhole imaging and the perspective projection
- 2.6.2 Homogeneous coordinates
- 2.6.3 Camera in a general configuration
- 2.6.4 Intrinsics, extrinsics, and what cropping really does
- 2.6.5 What perspective preserves, and what it destroys
- 2.6.6 Wide-angle distortion: spheres bulge and faces stretch at the edges
- 2.6.7 Photography with focal length: framing, magnification, and compression
- 2.6.8 Depth, ray length, and unprojection
- 2.7 Lens image formation
- 2.8 Image measurements as integrals
- 2.8.1 The plenoptic function
- 2.8.2 The pixel integral
- 2.8.3 Exposure values (EV) and stops
- 2.8.4 Splitting the integral
- 2.9 Depth of field
- 2.9.1 The circle of confusion
- 2.9.2 Depth of field versus depth of focus
- 2.9.3 Choosing the maximum circle of confusion
- 2.9.4 The double cone, and the near and far limits
- 2.9.5 Hyperfocal distance
- 2.9.6 The surprising invariance
- 2.9.7 Background blur beyond the focus plane
- 2.9.8 Defocus is not a blur of the image
- 2.9.9 Sensor size scales depth of field
- 2.10 Motion blur
- 2.10.1 Blur from subject motion
- 2.10.2 Blur from camera shake
- 2.10.3 How shift-invariant is camera-shake blur?
- 2.10.4 Which dominates: translation or rotation?
- 2.10.5 The hand-holding rule of thumb
- 2.10.6 Shake from the shutter and mirror
- 2.10.7 Panning: fighting subject motion by moving the camera
- 2.10.8 Freezing motion with light
- 2.11 Sensors: photosites, CCD vs CMOS
- 2.11.1 The photoelectric effect
- 2.11.2 Photon to number: the photosite
- 2.11.3 Spectral sensitivity: matching the eye, catching photons, and white balance
- 2.11.4 Beyond the visible spectrum: near-infrared, thermal, and ultraviolet
- 2.11.5 Analog-to-digital conversion
- 2.11.6 CCD versus CMOS
- 2.11.7 Time integration: the exposure
- 2.11.8 Shutters
- 2.11.9 A taste of modern sensor tricks
- 2.11.10 The impact of sensor size
- 2.11.11 How far sensors have come
- 2.12 Sensing color: multiplexing strategies
- 2.12.1 Multiplexing in time
- 2.12.2 Multiplexing in space: the color filter array
- 2.12.3 Multiplexing across separate sensors: the beam-splitter
- 2.12.4 Multiplexing per pixel by dispersion: color routers and nano-prisms
- 2.12.5 Multiplexing per pixel by tunable absorbers: quantum dots
- 2.12.6 Multiplexing in depth: stacked photodiodes
- 2.12.7 Hybrid strategies
- 2.12.8 Beyond trichromatic capture: full spectrum and multispectral
- 2.13 Noise, signal-to-noise ratio and dynamic range
- 2.14 Imaging as a linear system
- 2.15 Imaging as an inverse problem
- 2.15.1 Linear algebra: how hard is this problem?
- 2.15.2 Priors and the manifold of natural images
- 2.15.3 Probability: the Bayesian view
- 2.15.4 Optimization, inference, loss
- 2.15.5 When you can design the operator — and a teaser of information theory
- 2.15.6 Recap: what makes imaging hard, and where it's going
- 2.15.7 Harder inverse problems: factorization
- 2.16 Limitations of the medium
- 2.17 Displays
- 2.17.1 The range of displays
- 2.17.2 Display technology
- 2.17.3 HDR displays and dual modulation
- 2.17.4 The film movie projector
- 2.17.5 Printing
- 2.17.6 Why the display characteristics matter
- 2.17.7 Distance, resolution and acuity
- 2.17.8 Sharpening for the display: size and distance
- 2.17.9 Light level viewing conditions
- 2.17.10 Robustness of perspective to the viewer's viewpoint
- 2.17.11 Gamut and gamut mapping
- 2.17.12 Color management, ICC, and industry standards