draft · v0.1.241
💬Comments welcome. To leave a note, select any text and click the note / highlight button that pops up — or open the panel with the tab at the top-right (‹). Notes are visible only inside our private review group.
Computational Photography, an AI-powered Slopendium — 04 Photography
expand to📖 Full book outline1 parts · 13 chapters · 47 sections · 42 figures embedded · 11 placeholders · double-click a figure to enlarge
Part 4 PHOTOGRAPHY
fig-exposure-triangle
fig-exposure-triangle · exposure triangle 🟨
Fundamentals followed light from a source to a perceived color. This part turns to the act of photography itself: how the abstractions of exposure, focus, and focal length become the controls, modes, and hardware of a real camera, and how a phone reaches the same goal by a computational route. It closes by dismantling the myth that the camera is objective. Every "correction," "enhancement," and "restoration" in the rest of the book is a choice with a default, not a neutral truth, and that reframing is the bridge into the computational chapters that follow.
• **Exposure** — shutter, aperture, ISO, stops (Big lesson L3.1), metering and the 18% grey, priority modes, ETTR, and the PASM/auto-ISO mode dial.
• **Lenses** — focal length and field of view, prime vs zoom, fast vs slow, image stabilization, keeping the glass clean, and filters (polarizer, ND, graduated ND).
• **Focus, autofocus, and depth of field** — CDAF, PDAF, on-sensor dual-pixel, depth-from-defocus, focus modes, and subject/eye detection.
• **Cameras** — the viewfinder, the anatomy of a mirrorless body and of a phone, camera vs phone, the taxonomy of camera types, cameras that measure rather than photograph, and the non-imaging sensor suite.
• **Video** — frame rate and shutter angle, rolling shutter, log profiles, codecs.
• **Illumination and the flash** — goals of lighting; natural illumination (time of day, weather, sky models, golden/blue hour); flash metering (TTL, sync speed and HSS, fill flash); indirect illumination and bounce; multiple-point lighting.
• **Traditional and Digital Darkroom** — the chemical darkroom (develop and print, dodging and burning, the Zone System) and the digital one (parametric raw developers vs pixel editors), every tool an algorithm studied later.
• **Photographs are usually not passive objective recordings** — the photographer's degrees of freedom, why faithful is not the same as realistic, and the continuum from adjustment to fabrication.
• **Types of photography** — a genre survey (portrait, landscape, wildlife, sport, macro, night, astro, product, and more), each a point in the space of subject/purpose/setting/light/technique, with the challenge, success criteria, and settings it drives.
• **Photography and videography jobs** — the field as a set of careers, mapped along three axes (market, function, employment) so a job is a triple, and how computational photography is collapsing the crew back toward the individual.
4.1 Exposure
• chapter intro: having built the physics, we now sit behind a real camera — its **controls, modes, and guts** — and see how phones reach the same goal by a wholly different (computational) route
4.2 Lenses
• chapter intro: what a focal length is *for*, fast vs slow, prime vs zoom, image stabilization, keeping the glass clean, and the filters that change the captured light.
4.3 Focus, autofocus, and depth of field
fig-af-families
fig-af-families · autofocus families: contrast-detect hill-climb · phase-detect sub-aperture offset · on-sensor/dual-pixel PDAF
fig-telephoto-vs-retrofocus
fig-telephoto-vs-retrofocus · two two-group schematics — telephoto (+ then −, principal plane H′ pushed in front → physical length < f) vs retrofocus/inverted-telephoto (− then +, long back-focal distance to clear the SLR mirror); marks f vs physical length, H′, F′
⬜ figure not yet created
phase-detection geometry — in-focus vs too-near vs too-far [fig-pdaf-phase fig-pdaf-phase
fig-face-landmarks
fig-face-landmarks · face analysis detect→landmark progression: (a) detection box, (b) classic 68-point landmark layout, (c) dense ~468-point surface mesh, over a schematic face (Face tracking, 7.x)
• **the two classic passive AF families**:
• **contrast-detection (CDAF)**: drive the lens until image **contrast/sharpness is maximal** — accurate but has to **hunt** (it doesn't know which way is "more in focus", only that it overshot); historically slow, common on early mirrorless / compacts
• **phase-detection (PDAF)**: look at the scene through **two opposite edges of the lens** (sub-apertures) → two slightly shifted images; the **phase difference gives the direction *and* amount** of defocus in **one shot**, so the lens moves straight to focus (open-loop, no hunt). Classic SLR mechanism (a dedicated AF sensor under the mirror); it is literally a **tiny stereo measurement**
• **on-sensor PDAF / dual-pixel**: mirrorless bodies put phase detection *on the imaging sensor* — masked AF pixels, or **dual-pixel** designs that split each photosite into two halves that see opposite sub-apertures → phase detection at (nearly) every pixel, the best of both worlds (used for Pixel's portrait-mode depth too — a literal micro-stereo pair)
• **depth-from-defocus AF (Panasonic DFD)**: a third route — model how the lens's blur (PSF) changes with focus and read the **defocus direction and amount** from **two frames at slightly different focus**, giving a PDAF-like one-step cue from a plain **contrast** sensor (no phase pixels); rides on CDAF, needs a **per-lens blur profile** (full treatment → Optics, *Focus*)
• **active AF** (for completeness / history): time-of-flight ultrasonic (old Polaroid sonar) and IR rangefinding — work in the dark but limited; distinct from an **AF-assist lamp** that merely adds contrast for passive AF
• **focus modes**: **AF-S / one-shot** (lock once, for still subjects) vs **AF-C / AI-Servo** (track continuously, predict motion, for action); plus **MF** with focus aids (peaking, magnify — see UI)
• **focus selection & automation** — where computation now lives: single-point / zone / wide-area selection, then **subject-detection AF** — **face → eye → animal/bird/vehicle** detect that picks and **tracks** the subject (deep-learning driven; →ML). **Eye-AF** is the headline modern feature for portraits.
• **handling tricks**: **back-button focus** (decouple AF from the shutter button — focus with a thumb button, shutter only releases) and **focus-and-recompose** (lock focus on the subject, then reframe — beware the small focus-plane error at wide apertures / close range)
4.4 Cameras
• chapter intro: the viewfinder and its overlays, the anatomy of a mirrorless body and of a phone, camera vs phone, the taxonomy of camera types, cameras that measure rather than photograph, and the non-imaging sensor suite.
4.4.4 Camera versus phone
4.5 Video
fig-shutter-angle
fig-shutter-angle · shutter angle sets the blur — a rotating-disc shutter at $0°/180°/360°$ admitting a smaller/larger fraction of the frame interval $T$; $180°$ ($\tau=T/2$) is the cinematic film-look, small angles strobe 🟨
fig-rolling-shutter-skew
fig-rolling-shutter-skew · rolling-shutter distortion — a global shutter keeping a vertical pole upright under a fast pan vs a rolling shutter where per-row readout $t(r)=t_\text{frame}+r\,t_\text{row}$ shears the pole and smears fan blades (the jello effect) 🟨
• **frame rate & shutter angle**: video sets exposure time as a fraction of the frame interval — the **180° shutter-angle** convention (shutter ≈ ½ the frame time) gives "natural" motion blur; high frame rate → slow motion
• **rolling shutter / "jello"**: CMOS reads the sensor **row by row**, so fast motion (or a fast pan, or a spinning prop) **skews/wobbles** — a **global shutter** avoids it but is rarer/costlier (ties back to CMOS readout in Image measurements as integrals)
• **log / flat profiles + grading**: shoot a **log** (flat, low-contrast) profile to preserve dynamic range, then **color-grade** in post — the video cousin of raw + tone curve
• **codecs & bitrate**: intra- vs inter-frame compression, 8- vs 10-bit, chroma subsampling (4:2:0 vs 4:2:2), bitrate — trade file size vs editing/grading headroom
• *(kept deliberately brief — the algorithms and the full motion treatment are in **Motion/Video**)*
4.6 Illumination and the flash
fig-point-op-levels
fig-point-op-levels · levels on a real (flat/hazy) photo: bunched-up luma histogram stretched out to fill [0,1] by setting a black point and a white point — input + after + transfer curve with the clip-and-stretch anchors and the two histograms (Point operations → Black point, white point, and levels, BASIC)
fig-wide-angle-perspective-distortion
fig-wide-angle-perspective-distortion · wide-angle perspective distortion: off-axis spheres image as radially-stretched ellipses (flat-sensor geometry), and faces near a wide frame's edge are widened — not a lens flaw; forward-ref to correction (Pinhole image formation)
fig-flash-sync
fig-flash-sync · flash & focal-plane-shutter sync: whole frame ≤ X-sync · slit/partial frame above it · high-speed-sync pulse train; fill-flash noted
fig-telephoto-vs-retrofocus
fig-telephoto-vs-retrofocus · two two-group schematics — telephoto (+ then −, principal plane H′ pushed in front → physical length < f) vs retrofocus/inverted-telephoto (− then +, long back-focal distance to clear the SLR mirror); marks f vs physical length, H′, F′
⬜ figure not yet created
**natural illumination from a physical sky, 3-D interactive [fig-sky-illumination-sim fig-sky-illumination-sim
fig-dof-raydiagram
fig-dof-raydiagram · **interactive** 2-D thin-lens depth-of-field ray diagram + plot: set focal length, f-number, subject distance, circle of confusion and sensor size; the diagram shows the focused cone landing on the sensor and a background point spreading into a blur disk; reads off near/far limits, hyperfocal distance H and magnification; "constant magnification" toggle plots total DoF vs focal length (nearly flat — at fixed framing DoF barely depends on focal length). A lighter companion to the 3-D dof sim.
• **Natural illumination**: sun (hard, small) + sky (soft, huge dome) and their changing ratio; time of day (noon toplight → warm low sun; **golden hour**, **blue hour**); weather (overcast = giant diffuser; haze/aerial perspective); **sky models** (CIE distributions; Perez all-weather, Preetham daylight, Hošek–Wilkie), tuned by turbidity.
• 🖱️ **Interactive (web edition):** a 3-D **natural-illumination / physical-sky simulator** — a head/figure under a modeled sky dome; controls for sun **azimuth/elevation** and **turbidity**; a toggle shows the sky **approximated by many point lights** over the hemisphere (plus a sun key). [fig-sky-illumination-sim; interactive]
• **TTL metering**: the camera fires a **pre-flash**, meters the return **through the lens**, and sets flash power automatically — the flash analog of auto-exposure
• **sync speed & high-speed sync**: a **focal-plane** shutter only fully uncovers the sensor up to its **X-sync speed** (~1/200 s) — faster than that and a moving slit means the flash can't light the whole frame; **high-speed sync (HSS)** pulses the flash rapidly to cover the slit (at a big power cost). A **leaf shutter** (in-lens) opens fully at *any* speed → **syncs at all shutter speeds** (a real advantage for daylight fill).
• **fill flash**: in harsh/backlit light, add flash at **−1 to −2 EV** (the slides say −1.5 to −2, i.e. 3–4× below ambient) to **open the shadows** without looking flashed — it's **dynamic-range management**, not "more light"
• **bounce & off-camera**: never bare flash head-on (flat, harsh, red-eye) — **bounce** off a ceiling/wall (~45°) or use a diffuser to enlarge the source → softer light; **off-camera** flash gives directional, shaped light (→ Computational illumination; computational bounce flash, Davis SIGA 2016)
• 🖱️ **Interactive (web edition):** a live **portrait-lighting simulator** — three soft area lights (key / fill / kicker, each with azimuth, elevation, size/softness, intensity, and colour) on a 3D face, with **area-light-supersampled soft shadows** and a physiological melanin/hemoglobin skin model; a from-behind "setup" view shows the lights as studio umbrellas plus the camera, and lighting presets (butterfly, loop, Rembrandt, split, clamshell, rim) let you feel how light *placement and size* sculpt a face. [fig-portrait-lighting-sim; interactive] *(3D studio umbrella generated with Meshy AI — acknowledge in the caption)*
4.7 Traditional and Digital Darkroom
4.8 Limitations of the medium
4.9 Displays
• chapter intro: the last link in the chain. The range of displays (phone, laptop, TV, projector, print) and how their characteristics — size and viewing distance, resolution, dynamic range, gamut, and viewing environment — shape both the experience and the processing. This chapter also carries color management: the ICC workflow and the industry standards that keep color consistent across capture, editing, and display. Forward-reference to [[Integral and immersive imaging]] for near-eye and immersive displays.
4.10 Photographs are usually not passive objective recordings
4.11 Types of photography
• **the five axes** — subject (portrait, landscape, wildlife…), purpose (art, commercial, journalistic, scientific, personal), setting (studio vs location, indoor vs outdoor), light regime (daylight, low-light/night, mixed artificial, controlled), technique/apparatus.
• for each genre: the **challenge**, the **criterion for success**, and the **lens / aperture / shutter / ISO / light / support** it drives.
4.12 Photography and videography jobs
• **thesis**: roles organize along three roughly independent axes, so a job is a triple; the triple also predicts the technical demands, tying the profession back to the book's chapters.
4.13 Why people take photos
• **thesis**: motive and machine shape each other; the medium's shift from a memory technology to a communication one is the spec the computational pipeline has been meeting all along. The "why / what-kinds" is well studied (HCI + psychology); the "when" is mostly a laundered vendor survey, yet it is directly computable from public corpora.