How does Portrait Mode fake a blurry background?

Why phones need to fake it. Background blur from optics depends on the width of the lens opening (the entrance pupil). At the same framing, the blur of a distant background, as a share of the frame, scales with that width. A 50 mm lens at f/1.8 opens 28 mm wide; a phone’s main lens is only about 6–7 mm long (a 24–26 mm full-frame equivalent on a much smaller sensor), so at f/1.8 it opens only about 3.5 mm wide and blurs roughly 8× less. That is why phone photos look sharp from front to back.

How Portrait Mode works. The phone estimates a depth map: how far away each part of the picture is. Phones get depth from two cameras (stereo), from dual-pixel autofocus sensors (each pixel sees through the left and right halves of the lens), from lidar on some models, and from neural networks that find people and estimate depth. Software then blurs each part of the picture according to its distance from the chosen focus distance, often shaping bright lights into lens-like discs.

This model. Blur radius r = R · (2.8 / N) · |1/d − 1/d_focus|, the thin-lens rule that defocus grows with the difference in inverse distance. R = 24 px of a 1600-px-wide photo per 1/m at the f/2.8 look, and N is an f-number “look”, as on a phone’s depth slider (here f/1.4–f/16). Readouts give the blur disc’s diameter as a share of the frame width. “A phone lens alone” uses 8× less blur at the same focus (the 28 mm vs 3.5 mm openings above).

The depth map is gray (white = near, black = far). Gray is turned into distance with anchors chosen for this scene (her face 1 m, the string lights 3.5 m, the street 14 m, the trees 30 m). The phone-like map is used at 1/6 of the photo’s resolution, as phone depth maps are much coarser than the photo; the “fine” setting uses the full-resolution estimate. She is treated as one layer at 1 m, cut out by the depth map, and laid over the background; the background behind her is filled with a smooth guess. The image is pre-blurred at 9 levels with a Gaussian of σ = r/2, and each pixel blends the two levels around its blur; bright bulbs are spread into discs. Real phones use more careful rendering, edge refinement and noise matching, and their exact methods are not public.

Stray hairs. Hair strands are thinner than a pixel of a coarse depth map, so strands outside its edge are blurred as if they were background, and bits of background inside its edge stay sharp. Edges of hair, glasses and fuzzy outlines are the classic giveaways of synthetic blur.

The photo and its depth map are AI-generated illustrations (the depth map is an AI estimate of that photo); all blur and every number come from the model.

Sources: Google Research, Portrait mode on the Pixel 2 and Pixel 2 XL smartphones (2017); N. Wadhwa et al., Synthetic Depth-of-Field with a Single-Camera Mobile Phone, ACM Transactions on Graphics (SIGGRAPH 2018); Depth of field; 35 mm equivalent focal length; Bokeh.