How can a camera tell how far away something is?

Hold up a thumb and close one eye, then the other. Your thumb jumps, but the far wall barely moves. Your two eyes are making a measurement. A pair of cameras can do the same, and turn the jump into a distance.

▶ Try it yourself in Depth Lab: drag the cyclist toward the cameras and watch the shift between their two views grow.

What is the shift?

Two identical cameras sit side by side, a baseline B apart, both pointing straight ahead. A point at distance Z lands d = f · B / Z pixels farther right in the left camera’s image than in the right one. That shift is the disparity, so the distance is Z = f · B / d.

Here f is the focal length in pixels. The model camera has a 44-degree horizontal field of view, so at 1,280 pixels wide, f is about 1,584 pixels.

Why does closer mean a bigger shift?

Cameras 30 centimeters apart see a cyclist 10 meters away shift by about 48 pixels. At 3.6 meters the shift is about 132 pixels. A skyline 5 kilometers away shifts by only about 0.1 pixels. The shift falls off with distance, so distant things all look alike.

The tool’s measured distance rounds the shift to the nearest quarter pixel. A shift under one eighth of a pixel rounds to zero, and then the distance cannot be told from the skyline.

How good is the measurement?

A stereo matcher finds the shift only to within a fraction of a pixel. The model assumes plus or minus 0.25 pixels, which is illustrative. The distance uncertainty grows with the square of the distance, roughly δZ ≈ Z² · δd / (f · B). Doubling the distance quadruples that uncertainty. A wider baseline or a sharper camera shrinks it.

Predict first, then try

Before you drag, predict this: if the cyclist moves from 10 meters to 3.6 meters, will the shift grow by about 2.8 times, or by less? Then check it in Depth Lab. Compare the two shift values the tool reads out.

Next, move the cyclist back toward the skyline and watch the shift fall toward zero. Explain to yourself why a distant wall can look the same as a farther one.

Some driver-assistance systems use a forward-facing pair of cameras behind the windshield. Subaru’s EyeSight, for example, uses two cameras about a foot, or 30 centimeters, apart near the rear-view mirror.

What does the model leave out?

The model’s tag reads “ideal camera pair, flat road.” It is a flat road with two perfect pinhole cameras. The model leaves out:

• Calibration. Lens distortion and calibration, including rectification, which lines the two images up row by row.
• Matching. How a matcher finds the same point in both images. Smooth, shiny or repeating surfaces are hard.
• Other senses. Radar, single-camera cues and machine learning that many systems combine with stereo.
• Human vision. Your brain also turns your eyes and fuses the two views. People also use focus, motion parallax and familiar sizes.

The photos are AI-generated illustrations. The left and right views are made from one photo, shifted by the model’s disparity. Every number comes from the model.

Sources: Szeliski, Computer Vision: Algorithms and Applications, 2nd ed. (2022); Wikipedia, Computer stereo vision; Wikipedia, Binocular disparity; Wikipedia, Pupillary distance; Brake and Front End, Understanding Subaru’s EyeSight System; Zhang et al. (2020), Du²Net: Learning depth estimation from dual-cameras and dual-pixels; Franke (2013), stereo vision for driver assistance.