mediacodec.dev

Chroma subsampling: 4:2:0 vs 4:2:2 vs 4:4:4

Why video throws away three quarters of its colour information, why you almost never notice, and when you absolutely do.

Updated

Almost every video you have ever watched discards 75% of its colour information before compression even begins. This is chroma subsampling, and it is the oldest, most effective trick in video engineering: your eyes are sharp about brightness and blurry about colour, so video keeps brightness sharp and lets colour ride along at quarter resolution.

Luma, chroma, and why RGB leaves the building

Cameras capture RGB, but codecs immediately convert to YCbCr: one luma plane (Y: brightness, essentially a black-and-white image) and two chroma planes (Cb and Cr: colour differences). The point of the separation is that the two kinds of information can now be treated differently, and the human visual system says they should be. Retinal hardware dedicates far more resolution to luminance edges than to colour boundaries. Halve the resolution of the colour planes and almost nobody can see the difference on natural footage.

Reading the notation

The J:a:b notation describes how many chroma samples survive per J luma samples, across two rows:

Scheme Chroma resolution Colour data kept Where you meet it
4:4:4 Full 100% Screen capture masters, colour grading, PC-to-display links
4:2:2 Half horizontal 50% Professional cameras, broadcast production chains
4:2:0 Half both axes 25% Effectively all delivery: web video, streaming, Blu-ray, phones

4:2:0 gives each 2×2 block of pixels a single colour sample. That is the format of essentially everything consumers watch, and the bandwidth saving (half the raw data before the codec does any work) is a large part of why digital video is feasible at all.

When 4:2:0 hurts

Subsampling was tuned for photographs of the world, and the world cooperates: natural images rarely put saturated colour edges exactly one pixel apart. But point it at synthetic content and the seams show:

  • Small text and UI. Red text on a blue background at 4:2:0 gets smeared colour fringes: each 2×2 block gets one colour vote. This is why screen recordings of terminals and editors look mushy at settings that make film look great.
  • Saturated edges. A pure red title on black bleeds at its boundary; the luma edge is sharp but the colour edge is soft.

If you are encoding screen content for critical viewing, that is the moment to reach for 4:4:4 (-pix_fmt yuv444p in x264), with the caveat that hardware decoders and browsers widely assume 4:2:0, so 4:4:4 delivery belongs in controlled environments, not on the open web.

The even-dimensions rule

Subsampling is also why encoders demand even widths and heights: with one chroma sample per 2×2 luma block, an odd dimension leaves half a block: there is no way to represent a 719-pixel-wide frame’s final column of chroma. It is why FFmpeg’s scale filter has the -2 idiom (scale=1280:-2, used in our resize builder) and why width not divisible by 2 is one of the most-searched FFmpeg errors.

What to remember

For camera footage, 4:2:0 is the right call and nobody will ever see the difference; every streaming service on earth agrees. For screen content with fine text, subsampling is the first thing to suspect when output looks smeared. And when a pipeline mysteriously refuses odd dimensions or produces shifted colours after a resize, you are looking at chroma geometry, not a bug in your command.

Subsampling is one layer of the colour pipeline. The tags that say how the remaining values should be interpreted are the other, and they cause their own distinctive failure: see colour spaces, and why your video looks washed out.