Containers vs codecs: MP4, WebM and MKV

“What format is that video?” has two answers, and mixing them up is behind a surprising share of playback bugs. MP4 is not a codec. H.264 is not a file type. One is the wrapper; the other is the compressed data inside it.

What a container actually does

A container holds one or more streams and the bookkeeping that makes them playable together:

• Interleaving. Video and audio are chopped into small pieces and alternated, so a player reading the file front to back always has both.
• Timestamps. Every piece carries a presentation time. This is what keeps lips in sync.
• An index. A table mapping timestamps to byte offsets, which is what makes seeking possible without reading the whole file.
• Everything else. Subtitles, chapters, cover art, language tags, rotation metadata.

None of that touches the compressed video. The codec decides how a frame is turned into bits; the container decides how those bits are filed.

What goes in what

Container	Video codecs you’ll meet	Audio	Where it plays
MP4	H.264, H.265, AV1	AAC, Opus	Everywhere, including hardware players and old devices
WebM	VP9, AV1	Opus, Vorbis	All current browsers; a Matroska subset, royalty-free by design
MKV	Anything at all	Anything at all	Desktop players; not a browser format
MOV	ProRes, H.264	PCM, AAC	Editing and mastering, Apple-leaning

MKV is the permissive one: it will carry practically any codec, which makes it great for archives and useless for the web. MP4 is the opposite, a narrower set with near-universal support. WebM exists because MP4’s ecosystem grew up around patent-encumbered codecs, and the open web wanted a stack it could ship without licensing anybody’s portfolio.

The extension has to match the contents

A container will not carry anything you like. Put an Opus stream in a .m4a and most players choke, because the extension sets an expectation the file does not meet. The rule that avoids it: the extension follows the codec, not your preference. AAC goes to .m4a, MP3 to .mp3, Opus to .opus, FLAC to .flac. The audio extractor picks the right one for whichever codec you choose, which is the whole reason it asks.

Remuxing is free, re-encoding is not

Because the container and the codec are separate layers, you can very often change the wrapper and leave the compressed data untouched. This is remuxing, and it takes about as long as copying the file:

# MKV to MP4 with no re-encode: seconds, and bit-for-bit identical video
ffmpeg -i input.mkv -c copy output.mp4

-c copy says “do not decode anything, just re-file it.” No quality is lost because no pixels are touched. The same trick is why stream-copy trimming is instant while a frame-accurate cut has to re-encode.

The catch is that remuxing only works when the destination container accepts the codec. -c copy from MKV to MP4 succeeds if the video inside is H.264 and fails if it is VP9, and FFmpeg will tell you so rather than guess. That error is the layer boundary making itself visible.

Fragmented MP4, and why streaming changed the shape

A normal MP4 keeps its index in one lump, traditionally at the end of the file. That is fine for a download and hopeless for streaming, where a player needs to start before it has the whole thing. Two fixes exist. Moving the index to the front (-movflags +faststart) makes progressive playback work. Splitting the file into self-contained fragments, each with its own small index, makes adaptive streaming work. Fragmented MP4 is the second one, and it is what modern HLS and DASH both ship.

Finding out what you actually have

ffprobe -v error -show_entries stream=index,codec_type,codec_name,profile \
  -of default=noprint_wrappers=1 input.mp4

Do this before debugging a playback problem. Roughly half the time the answer is that the file contains a codec the target cannot decode, and the container was never the issue. If the target is a browser, our capability report tells you what that browser can actually handle, and the transcode lab will rewrap or re-encode a file without uploading it anywhere.

What to remember

The extension tells you the wrapper and nothing about the contents. Changing the wrapper is cheap; changing the contents costs an encode. When something will not play, find out which layer is at fault before reaching for a conversion, because half those conversions are unnecessary.