draft · v0.1.226
💬Comments welcome. To leave a note, select any text and click the note / highlight button that pops up — or open the panel with the tab at the top-right (‹). Notes are visible only inside our private review group.
Computational Photography, an AI-powered Slopendium — 03 Photography
expand to📖 Full book outline1 parts · 9 chapters · 22 sections · 41 figures embedded · 11 placeholders · double-click a figure to enlarge
Part 3 PHOTOGRAPHY
fig-exposure-triangle
fig-exposure-triangle · exposure triangle 🟨
Fundamentals followed light from a source to a perceived color. This part turns to the act of photography itself: how the abstractions of exposure, focus, and focal length become the controls, modes, and hardware of a real camera, and how a phone reaches the same goal by a computational route. It closes by dismantling the myth that the camera is objective. Every "correction," "enhancement," and "restoration" in the rest of the book is a choice with a default, not a neutral truth, and that reframing is the bridge into the computational chapters that follow.
• **Exposure** — shutter, aperture, ISO, stops (Big lesson L3.1), metering and the 18% grey, priority modes, ETTR, and the PASM/auto-ISO mode dial.
• **Lenses** — focal length and field of view, prime vs zoom, fast vs slow, image stabilization, keeping the glass clean, and filters (polarizer, ND, graduated ND).
• **Focus, autofocus, and depth of field** — CDAF, PDAF, on-sensor dual-pixel, depth-from-defocus, focus modes, and subject/eye detection.
• **Cameras** — the viewfinder, the anatomy of a mirrorless body and of a phone, camera vs phone, the taxonomy of camera types, cameras that measure rather than photograph, and the non-imaging sensor suite.
• **Video** — frame rate and shutter angle, rolling shutter, log profiles, codecs.
• **Illumination and the flash** — goals of lighting; natural illumination (time of day, weather, sky models, golden/blue hour); flash metering (TTL, sync speed and HSS, fill flash); indirect illumination and bounce; multiple-point lighting.
• **Traditional and Digital Darkroom** — the chemical darkroom (develop and print, dodging and burning, the Zone System) and the digital one (parametric raw developers vs pixel editors), every tool an algorithm studied later.
• **Programmability, or the lack thereof** — closed firmware, CHDK/Magic Lantern, the phone camera APIs, and why computation lives on phones.
• **Photographs are usually not passive objective recordings** — the photographer's degrees of freedom, why faithful is not the same as realistic, and the continuum from adjustment to fabrication.
3.1 Exposure
• chapter intro: having built the physics, we now sit behind a real camera — its **controls, modes, and guts** — and see how phones reach the same goal by a wholly different (computational) route
3.2 Lenses
• chapter intro: what a focal length is *for*, fast vs slow, prime vs zoom, image stabilization, keeping the glass clean, and the filters that change the captured light.
3.3 Focus, autofocus, and depth of field
fig-af-families
fig-af-families · autofocus families: contrast-detect hill-climb · phase-detect sub-aperture offset · on-sensor/dual-pixel PDAF
fig-telephoto-vs-retrofocus
fig-telephoto-vs-retrofocus · two two-group schematics — telephoto (+ then −, principal plane H′ pushed in front → physical length < f) vs retrofocus/inverted-telephoto (− then +, long back-focal distance to clear the SLR mirror); marks f vs physical length, H′, F′
⬜ figure not yet created
phase-detection geometry — in-focus vs too-near vs too-far [fig-pdaf-phase fig-pdaf-phase
fig-face-landmarks
fig-face-landmarks · face analysis detect→landmark progression: (a) detection box, (b) classic 68-point landmark layout, (c) dense ~468-point surface mesh, over a schematic face (Face tracking, 7.x)
• **the two classic passive AF families**:
• **contrast-detection (CDAF)**: drive the lens until image **contrast/sharpness is maximal** — accurate but has to **hunt** (it doesn't know which way is "more in focus", only that it overshot); historically slow, common on early mirrorless / compacts
• **phase-detection (PDAF)**: look at the scene through **two opposite edges of the lens** (sub-apertures) → two slightly shifted images; the **phase difference gives the direction *and* amount** of defocus in **one shot**, so the lens moves straight to focus (open-loop, no hunt). Classic SLR mechanism (a dedicated AF sensor under the mirror); it is literally a **tiny stereo measurement**
• **on-sensor PDAF / dual-pixel**: mirrorless bodies put phase detection *on the imaging sensor* — masked AF pixels, or **dual-pixel** designs that split each photosite into two halves that see opposite sub-apertures → phase detection at (nearly) every pixel, the best of both worlds (used for Pixel's portrait-mode depth too — a literal micro-stereo pair)
• **depth-from-defocus AF (Panasonic DFD)**: a third route — model how the lens's blur (PSF) changes with focus and read the **defocus direction and amount** from **two frames at slightly different focus**, giving a PDAF-like one-step cue from a plain **contrast** sensor (no phase pixels); rides on CDAF, needs a **per-lens blur profile** (full treatment → Optics, *Focus*)
• **active AF** (for completeness / history): time-of-flight ultrasonic (old Polaroid sonar) and IR rangefinding — work in the dark but limited; distinct from an **AF-assist lamp** that merely adds contrast for passive AF
• **focus modes**: **AF-S / one-shot** (lock once, for still subjects) vs **AF-C / AI-Servo** (track continuously, predict motion, for action); plus **MF** with focus aids (peaking, magnify — see UI)
• **focus selection & automation** — where computation now lives: single-point / zone / wide-area selection, then **subject-detection AF** — **face → eye → animal/bird/vehicle** detect that picks and **tracks** the subject (deep-learning driven; →ML). **Eye-AF** is the headline modern feature for portraits.
• **handling tricks**: **back-button focus** (decouple AF from the shutter button — focus with a thumb button, shutter only releases) and **focus-and-recompose** (lock focus on the subject, then reframe — beware the small focus-plane error at wide apertures / close range)
3.4 Cameras
• chapter intro: the viewfinder and its overlays, the anatomy of a mirrorless body and of a phone, camera vs phone, the taxonomy of camera types, cameras that measure rather than photograph, and the non-imaging sensor suite.
3.4.4 Camera versus phone
3.5 Video
fig-shutter-angle
fig-shutter-angle · shutter angle sets the blur — a rotating-disc shutter at $0°/180°/360°$ admitting a smaller/larger fraction of the frame interval $T$; $180°$ ($\tau=T/2$) is the cinematic film-look, small angles strobe 🟨
fig-rolling-shutter-skew
fig-rolling-shutter-skew · rolling-shutter distortion — a global shutter keeping a vertical pole upright under a fast pan vs a rolling shutter where per-row readout $t(r)=t_\text{frame}+r\,t_\text{row}$ shears the pole and smears fan blades (the jello effect) 🟨
• **frame rate & shutter angle**: video sets exposure time as a fraction of the frame interval — the **180° shutter-angle** convention (shutter ≈ ½ the frame time) gives "natural" motion blur; high frame rate → slow motion
• **rolling shutter / "jello"**: CMOS reads the sensor **row by row**, so fast motion (or a fast pan, or a spinning prop) **skews/wobbles** — a **global shutter** avoids it but is rarer/costlier (ties back to CMOS readout in Image measurements as integrals)
• **log / flat profiles + grading**: shoot a **log** (flat, low-contrast) profile to preserve dynamic range, then **color-grade** in post — the video cousin of raw + tone curve
• **codecs & bitrate**: intra- vs inter-frame compression, 8- vs 10-bit, chroma subsampling (4:2:0 vs 4:2:2), bitrate — trade file size vs editing/grading headroom
• *(kept deliberately brief — the algorithms and the full motion treatment are in **Motion/Video**)*
3.6 Illumination and the flash
fig-point-op-levels
fig-point-op-levels · levels on a real (flat/hazy) photo: bunched-up luma histogram stretched out to fill [0,1] by setting a black point and a white point — input + after + transfer curve with the clip-and-stretch anchors and the two histograms (Point operations → Black point, white point, and levels, BASIC)
fig-wide-angle-perspective-distortion
fig-wide-angle-perspective-distortion · wide-angle perspective distortion: off-axis spheres image as radially-stretched ellipses (flat-sensor geometry), and faces near a wide frame's edge are widened — not a lens flaw; forward-ref to correction (Pinhole image formation)
fig-flash-sync
fig-flash-sync · flash & focal-plane-shutter sync: whole frame ≤ X-sync · slit/partial frame above it · high-speed-sync pulse train; fill-flash noted
fig-telephoto-vs-retrofocus
fig-telephoto-vs-retrofocus · two two-group schematics — telephoto (+ then −, principal plane H′ pushed in front → physical length < f) vs retrofocus/inverted-telephoto (− then +, long back-focal distance to clear the SLR mirror); marks f vs physical length, H′, F′
⬜ figure not yet created
**natural illumination from a physical sky, 3-D interactive [fig-sky-illumination-sim fig-sky-illumination-sim
fig-dof-raydiagram
fig-dof-raydiagram · **interactive** 2-D thin-lens depth-of-field ray diagram + plot: set focal length, f-number, subject distance, circle of confusion and sensor size; the diagram shows the focused cone landing on the sensor and a background point spreading into a blur disk; reads off near/far limits, hyperfocal distance H and magnification; "constant magnification" toggle plots total DoF vs focal length (nearly flat — at fixed framing DoF barely depends on focal length). A lighter companion to the 3-D dof sim.
• **Natural illumination**: sun (hard, small) + sky (soft, huge dome) and their changing ratio; time of day (noon toplight → warm low sun; **golden hour**, **blue hour**); weather (overcast = giant diffuser; haze/aerial perspective); **sky models** (CIE distributions; Perez all-weather, Preetham daylight, Hošek–Wilkie), tuned by turbidity.
• 🖱️ **Interactive (web edition):** a 3-D **natural-illumination / physical-sky simulator** — a head/figure under a modeled sky dome; controls for sun **azimuth/elevation** and **turbidity**; a toggle shows the sky **approximated by many point lights** over the hemisphere (plus a sun key). [fig-sky-illumination-sim; interactive]
• **TTL metering**: the camera fires a **pre-flash**, meters the return **through the lens**, and sets flash power automatically — the flash analog of auto-exposure
• **sync speed & high-speed sync**: a **focal-plane** shutter only fully uncovers the sensor up to its **X-sync speed** (~1/200 s) — faster than that and a moving slit means the flash can't light the whole frame; **high-speed sync (HSS)** pulses the flash rapidly to cover the slit (at a big power cost). A **leaf shutter** (in-lens) opens fully at *any* speed → **syncs at all shutter speeds** (a real advantage for daylight fill).
• **fill flash**: in harsh/backlit light, add flash at **−1 to −2 EV** (the slides say −1.5 to −2, i.e. 3–4× below ambient) to **open the shadows** without looking flashed — it's **dynamic-range management**, not "more light"
• **bounce & off-camera**: never bare flash head-on (flat, harsh, red-eye) — **bounce** off a ceiling/wall (~45°) or use a diffuser to enlarge the source → softer light; **off-camera** flash gives directional, shaped light (→ Computational illumination; computational bounce flash, Davis SIGA 2016)
• 🖱️ **Interactive (web edition):** a live **portrait-lighting simulator** — three soft area lights (key / fill / kicker, each with azimuth, elevation, size/softness, intensity, and colour) on a 3D face, with **area-light-supersampled soft shadows** and a physiological melanin/hemoglobin skin model; a from-behind "setup" view shows the lights as studio umbrellas plus the camera, and lighting presets (butterfly, loop, Rembrandt, split, clamshell, rim) let you feel how light *placement and size* sculpt a face. [fig-portrait-lighting-sim; interactive] *(3D studio umbrella generated with Meshy AI — acknowledge in the caption)*
3.7 Traditional and Digital Darkroom
3.8 Programmability, or the lack thereof
fig-bokeh-shapes
fig-bokeh-shapes · bokeh: out-of-focus highlights take the aperture's shape — circular (many blades / wide open) vs polygonal (few blades, stopped down); synthetic defocus-disk fields + aperture insets
• most cameras run **closed firmware**: you cannot run your own code on them, so the **computational-photography ideas in this book can't be tried on a stock camera** — a real obstacle for experimentation
• the **hacks**: **CHDK** (Canon PowerShot) and **Magic Lantern** (Canon DSLRs) are community firmware add-ons that expose scripting, raw video, bracketing, intervalometers, and focus stacking — born exactly because the manufacturers wouldn't
• the **open road**: **Android's Camera2 / CameraX** API (and Google's earlier FCam research camera) expose per-frame control of exposure / focus / gain and access to **raw burst** — which is *why phones, not cameras, became the home of computational photography*
• the **maker road**: a **Raspberry Pi + camera module** (the HQ camera, driven by **libcamera / picamera2**) is a cheap, fully **open and programmable** camera — per-frame exposure/gain control and **raw** access on a hobbyist board — the natural platform for *trying this book's algorithms* and building DIY computational cameras
• **why this matters for the book**: our algorithms (HDR+, burst denoising, computational bokeh) need **programmable capture** — control of the per-frame integral and access to raw; the platform that grants it is where the research happens
3.9 Photographs are usually not passive objective recordings