2.13 Noise, signal-to-noise ratio and dynamic range🔗⧉

Noise earns a chapter of its own because, more than resolution or optics, noise is what ultimately limits our ability to measure and reconstruct an image. Almost everything else in this book — denoising, demosaicking, deblurring, HDR merging, depth from stereo, super-resolution — is at bottom an attempt to recover a clean signal from noisy measurements, and their performance is limited by how much noise corrupts the input. So understanding where noise comes from, how it scales with light, and which kind dominates when is not a side topic: it is the quiet constraint behind the whole field. It tells you what is recoverable and what is lost for good, how many photons or frames you need to reach a target quality, and — when a result disappoints — whether you are limited by your algorithm or by the physics of the measurement itself. Read this chapter as the floor under every later one.

2.13.1 Noise sources🔗⧉

Counting photons is what makes a sensor linear, but counting is also where noise comes from. Take two photos of the same still scene at the same settings and they are not byte-for-byte identical; each pixel value jitters from frame to frame. Several physical sources contribute, and because they are statistically independent, their variances add, so to combine them we add variances ($\sigma^2$), not standard deviations (Figure 2.13.3).

Photon (shot) noise is the deepest source, and it is not a defect of the sensor — it is the light itself. Light arrives as discrete photons at random times, so the count in a fixed interval fluctuates even from a perfectly steady source. The statistics are Poisson, and the defining property of a Poisson process is that its variance equals its mean (Hasinoff 2012 is the canonical short reference). So if a pixel collects $N$ photons on average, the variance of the count is $N$ and the standard deviation is

This is irreducible: no better sensor can remove it, because it is a property of light, not of electronics. It dominates the midtones and highlights, where $N$ is large. The only mitigation is averaging: average $N_f$ independent frames of the same scene and the shot noise drops by $\sqrt{N_f}$ — the statistical basis of burst denoising and astrophotography stacking.

The remaining sources are added by the hardware:

• Read noise is added by the amplifier and the analog-to-digital converter (ADC) at readout. It is signal-independent — a roughly fixed amount regardless of brightness — so it dominates the shadows (where the signal is tiny) and sets the sensor's noise floor.
• Thermal (dark-current) noise comes from electrons freed by heat rather than light. It is independent of the scene but grows with exposure time and temperature, which is why long exposures and astrophotography call for sensor cooling and dark-frame subtraction (photographing the dark current with the shutter closed and subtracting it).
• Fixed-pattern noise is per-pixel non-uniformity: slightly different gain or offset at each photosite, plus hot or stuck pixels and banding. Because it is structured rather than random — the same pattern in every frame — it reads as worse than random noise of the same magnitude, and it is removed by calibration (flat and dark fields).

The same five sources, as a reference table — what each varies with, where it dominates, and whether you can remove it:

We can see that noise directly. Shoot a burst of one static scene at high ISO and take, for every pixel, its variance across the frames: the result is a map of the noise itself, laid over the picture (Figure 2.13.4). The noise is not uniform — it is heavier in the brighter regions (shot noise) and along edges — which is the clue to the rule that follows.

2.13.2 Noise variance is affine in pixel brightness🔗⧉

Because the variances of these sources add, and because the dominant two scale so simply with the signal, the whole noise level follows a strikingly simple rule that we can measure directly from data.

Read the affine model the other way and it becomes a noise simulator: the quickest way to turn a clean image into realistic noisy data, for stress-testing a denoiser or building the synthetic training pairs this book leans on (Deep learning). The one rule that makes or breaks it is to work on a linear image, never a gamma-encoded one. Variance is affine in brightness only in linear light; add the noise after the tone curve and you inject the wrong amount in the shadows, exactly where it shows, and the result looks plainly fake. So, on a linear image with values $I\in[0,1]$, walk the pixels and give each a Gaussian kick whose variance the affine model prescribes (Algorithm 2.13.1):

for each pixel value I:                  # I is LINEAR, in [0, 1]  (not gamma-encoded)
    var   = a * I + b                    # affine model (L2.44): shot slope a, read floor b
    noise = gaussian(mean = 0, std = sqrt(var))
    I     = I + noise
    I     = min(I, 1)                    # clamp to full well: highlights clip at 1
    # I   = max(I, 0)                    # optionally clamp to 0 — but SKIP this if the
                                         #   simulated sensor adds a black-level pedestal (a
                                         #   constant), which legitimately lifts small negatives
# apply gamma / the display curve only AFTER this loop, never before

The two constants set how noisy the "camera" is, and they span a wide range with sensor size and ISO (values for $I$ scaled to $[0,1]$). A large full-frame sensor runs from about $a\approx10^{-5},\ b\approx10^{-9}$ at base ISO to $a\approx10^{-3},\ b\approx10^{-7}$ pushed to high ISO; a small phone sensor from about $a\approx10^{-4},\ b\approx10^{-8}$ at base ISO to $a\approx10^{-2},\ b\approx10^{-6}$ at high ISO. Smaller wells (the phone) and more amplification (high ISO) both steepen the slope $a$ and lift the floor $b$. This Gaussian kick is the fast approximation; for a more physical version that draws the shot term from a Poisson distribution in electrons and applies the sources in capture order, see the fuller recipe later in this section.

The affine model and the Gaussian shape are the ideal. Real recorded values break the ideal in one important way at the extremes.

That clipping caveat is the first crack in the affine model, and a larger one is worth planting now. The law $\sigma^2 \approx a\,I + b$ describes the noise as the sensor hands it over — the raw mosaic, straight off the ADC; it is a fact about the sensor, not about the final photograph. Run that raw through the computational-photography pipeline and the clean affine line bends. A burst merge (align-and-average $N$ frames, Denoising basics) drives the variance toward $1/N$ of a single frame, but unevenly — the robust merge keeps more frames in still regions than near moving edges — so the noise becomes spatially varying, and highlight clipping means it is no longer even affine in brightness. Demosaicking (Demosaicking) bends it too: an interpolated color is a weighted average of neighboring pixels, and averaging shrinks variance, so reconstructed pixels are quieter than the ones actually measured — by different amounts, pixel to pixel. After a few ISP stages the tidy 'gain·signal + read²' line is gone, and a downstream algorithm that still assumes it (a plain AWGN denoiser, say) is fitting a model the data no longer obeys.

Manufacturers know this and design around it: many sensors deliberately lift the black point before digitizing. They add a fixed offset — a "black level" or bias, a few hundred code values — to the signal, so the whole bell of the read noise, negative excursions included, lands above zero and is recorded rather than clipped (Canon, for instance, sits its black level around code 512 or 2048 on 14-bit raw; other makers similar). The raw file then carries an explicit black-level tag that the converter subtracts back out. Keeping the noise unclipped this way is exactly what lets noise-weighted averaging, flat/dark calibration, and shadow-lifting stay unbiased down in the darkest tones — at the cost of a sliver of headroom. One discipline follows: after subtracting the black level, near-black pixels can legitimately go slightly negative, and that must be preserved, not clamped, all the way until display, or the bias creeps right back in.

2.13.3 The algebra of noise🔗⧉

Because the sources are independent, their variances add — the single most useful fact for reasoning about noise, and worth turning into a tool. For $N$ independent measurements each of variance $\sigma^2$, the sum has variance $N\sigma^2$ (standard deviation $\sqrt N\,\sigma$), while the average has variance $\sigma^2/N$ (standard deviation $\sigma/\sqrt N$). That $1/\sqrt N$ is the engine of all averaging-based denoising (a whole chapter later). The same algebra settles a subtler, very practical question: is it better to measure light with one big pixel, or with several small ones and add them up?

Take a patch that collects $N$ photons. Capture it with one big pixel: it counts $N$ photons (shot variance $N$, since Poisson variance $=$ mean) and adds the read noise once, $\sigma_r^2$. Now split the same patch into four small pixels, each catching $N/4$ photons. Summing their four readings recovers the signal $N$, and the shot noise is identical — the four shot variances $N/4$ add back to $N$, exactly the big pixel's. But each small pixel contributes its own read noise, so the sum carries $4\sigma_r^2$: the read noise is paid four times over. So, provided no light is lost in the gaps between photosites, the big pixel's only real advantage is read noise, not shot noise. Shot noise depends solely on how many photons you caught, never on how you diced up the sensor; read noise is charged once per read-out.

This is exactly why binning matters, and why how you bin matters. Charge binning — pooling the pixels' charge in the analog domain before it reaches the amplifier and ADC — reads the combined charge once, paying read noise a single time and recovering the full big-pixel advantage; this is the trick a good quad-Bayer sensor uses in low light. Digital binning — reading the four pixels separately and adding the numbers afterward — pays read noise four times and captures only the shot-noise equivalence. The gap is invisible in bright light (shot-noise-dominated) and decisive in the dark (read-noise-dominated). It also restates the affine model $\sigma^2 = g\,I + \sigma_r^2$ from a new angle: aggregating $k$ measurements multiplies the read-noise term's contribution by $k$ unless you combine before read-out.

2.13.4 Signal-to-noise ratio: it's about ratios🔗⧉

Here is the fact that surprises people and ties the whole part together. Since shot noise is $\sigma = \sqrt{N}$, the absolute noise is actually larger in the bright parts of the image than in the dark parts — a highlight collecting 10,000 photons has $\sigma = 100$, while a shadow collecting 100 photons has only $\sigma = 10$. And yet noise looks worst in the shadows. The resolution is that perception cares about the signal-to-noise ratio (SNR), not the absolute noise. For shot noise,

The SNR grows with brightness, so it is worst where the light is faintest — the shadows — even though the raw noise magnitude is smallest there (Figure 2.13.8). Ratios are all that matters. This is the same lesson that drives gamma/log encoding (we encode so that equal ratios get equal steps) and exactly why exposing to the right works: collecting more photons everywhere, especially in the shadows, raises $N$ and therefore raises the SNR where it is worst.

One caution before we go on, because the acronyms collide: SNR is not PSNR. The signal-to-noise ratio here is a property of a capture — the true signal at a pixel relative to the sensor's noise there ($\sqrt N$ for shot noise) — and it answers how clean is this measurement? Peak signal-to-noise ratio (PSNR), which you meet whenever an algorithm is evaluated, is a different animal: a full-reference quality metric that compares a processed image $\hat I$ against a ground-truth reference $I$, computed from their mean-squared error as

where $\mathrm{peak}$ is the largest possible value (255 for 8-bit) and $\mathrm{MSE}=\tfrac1n\sum(\hat I-I)^2$. So SNR describes a photograph (how much noise a single measurement carries, with no reference needed), while PSNR scores a reconstruction against a known truth (how close a denoiser or codec got). They share "signal-to-noise" and a decibel scale and nothing else — one is a fact about the sensor, the other a yardstick for algorithms (and, as the metrics discussion later notes, a flawed one: because PSNR is just re-scaled MSE, it quietly rewards the blurry, safe answer).

The affine model settles a lot of everyday "will this help my noise?" questions at a glance. Read each move against $\sigma^2 = g\,I + \sigma_r^2$ (signal $I$, photon-gain slope $g$, read-noise floor $\sigma_r$), tracking what happens to the signal, the noise $\sigma$, and their ratio. The entries below are for the shot-noise-dominated regime (well-lit midtones); the read-noise term $\sigma_r$ changes only the two rows that mention it.

The pattern is stark: rescaling the numbers you already have (ISO, software exposure) never improves the signal-to-noise ratio; combining $N$ independent measurements improves it as $\sqrt N$; and the one thing that genuinely beats noise is collecting more light.

2.13.5 Dynamic range🔗⧉

Two extremes bound what a single exposure can hold. At the top, a pixel's well saturates: once it has counted its full-well capacity of electrons, more light adds nothing and the highlight clips to white. At the bottom, the read-noise floor sets the dimmest signal still distinguishable from noise. The ratio of the two is the sensor's dynamic range — how much of a high-contrast scene a single exposure can capture. When a scene's range exceeds the sensor's — a bright window and a dim interior in one frame — no single exposure holds both ends, which is the motivation for high-dynamic-range (HDR) imaging, merging several exposures (Multiple-exposure part) (Figure 2.13.9). Like everything else a photographer counts, dynamic range is quoted in stops — factors of two — which connects it straight to the exposure controls.

There is one more limit that no sensor spec sheet lists: the lens. Bright parts of a scene scatter stray light inside the lens and camera body — faint reflections off every element surface, the diaphragm blades, and the barrel walls — that settle as a veil of light spread across the whole frame. This veiling glare, the diffuse cousin of lens flare, lifts the darkest tones off true black, raising the effective noise floor from the optical side just as read noise raises it from the electronic side. It therefore caps the dynamic range a real photograph can reach below whatever the bare sensor could deliver, and it bites hardest exactly when dynamic range matters most, shooting a dim subject against a bright source. It is why lens coatings, hoods, and a clean front element buy real shadow contrast, and why the deepest blacks in a studio shot are easier to protect than those in a backlit scene. Flare and veiling glare get their full treatment as optical defects in Aberrations and optical challenges (§10.1).

How wide a range, exactly? It helps to put numbers on it, because the dynamic ranges of real media and of the eye differ by a lot (Figure 2.13.10). A color slide is narrow, only about 5–6 stops; a reflective print about 6–7; a film negative is famously forgiving at roughly 12–13 stops of latitude; a phone sensor manages about 10–12 and a full-frame sensor about 14. The human eye sees about 10–14 stops instantaneously, but allow it to adapt — the slow re-centering of the retinal response covered with perception — and it spans 20+ stops over time; some animals do better still within their niche. Against all of these, an outdoor sun-and-shadow scene can exceed 20 stops in a single frame — which is precisely why no single capture holds it, and why HDR exists.

Recovering an underexposed shot: full-frame vs. phone. The clearest way to feel that dynamic range scales with photosite size is to break one. Shoot the same underexposed scene as raw on a full-frame camera and on a phone, then push the exposure up in software by three or four stops (Figure 2.13.11). The full-frame frame cleans up — its deep full-well and low read-noise floor give it the headroom to be lifted — while the phone frame breaks into shadow noise and banding, because lifting the exposure also amplifies its read-noise floor, which sat much closer to the signal to begin with. That is a direct, visible demonstration that dynamic range scales with photosite (and sensor) size, and exactly why phones lean on burst capture and HDR+ (Multiple-exposure part) rather than trusting a single exposure. It is also a coding exercise — see Exercises & Experiments.

This is also the right moment to deflate a worry beginners often have about digital capture: that the image is quantized into a finite number of levels, and that this stair-stepping is what limits quality. It almost never is.

These three facts — that noise is the jitter of a photon count, that the signal-to-noise ratio is worst in the shadows, and that dynamic range is the full-well-over-floor a single exposure can hold — recur on nearly every later page: in how a photographer exposes to the right, in high-dynamic-range merging (Multiple-exposure part), and in the denoising of the basic pipeline. With measurement and its limits now in hand, the part turns next to what a second viewpoint buys you — depth — and then to the eye that will finally judge the result.

2.13.6 Regimes, and how close we are to the limits🔗⧉

Pulling the chapter together: which noise matters depends entirely on how much light you have, and a single picture usually spans several regimes at once.

• Bright light / highlights — shot-noise-limited. Plenty of photons, so shot noise ($\sqrt N$) dominates and read noise is negligible. SNR is excellent and only improves with more light; nothing but more photons (or a bigger sensor) helps.
• Deep shadows / low light — read-noise-limited. Few photons, so the constant read-noise floor dominates and sets where the image dissolves into mush. This is where sensor engineering still moves the needle and where computational averaging (bursts) buys real ground.
• Long exposures / heat — dark-current-limited. Time and temperature let thermal electrons pile up; cooling and dark-frame subtraction are the cure.

The same three regimes bound dynamic range, which is just full-well capacity ÷ read-noise floor (L2.48): the top set by how many electrons a pixel holds before clipping, the bottom by the read noise.

How close are we to the theoretical limits? On two of the three axes, modern sensors are already close to their physical limits.

• Quantum efficiency — the fraction of arriving photons actually converted to electrons — is already around 80–95 % in good back-illuminated sensors, so there is at most a few tenths of a stop of light-gathering left to win. In bright light we are essentially at the shot-noise limit: a perfect sensor would be only marginally cleaner, because the noise is in the light itself (L2.42).
• Read noise has fallen to ~1–2 electrons on the best CMOS, and specialized scientific CMOS and electron-multiplying sensors reach sub-electron read noise — within sight of counting individual photons (the single-photon avalanche diodes of the Advanced part do exactly that).
• Full-well capacity, and hence single-exposure dynamic range, is the axis still bound by pixel area and electrostatics — and it is precisely the one that has plateaued (the previous chapter's curves): a pixel of a given size can only hold so many electrons before it saturates.

So the photons themselves are nearly free and nearly all counted; what stays scarce is holding and reading them without adding noise, and there physics has almost run out of room. That is exactly why the frontier moved off the single capture and onto multi-frame computation — averaging down the floor and merging exposures to beat the well — the bridge from this chapter's physics to the rest of the book.

The whole budget is easiest to feel by turning the dials yourself (Figure 2.13.12).