Audio codecs: AAC, Opus, MP3 and the lossless ones

Audio is usually an afterthought in a video pipeline, which is unfortunate, because viewers forgive a soft picture and abandon a stream over bad sound. The good news is that the decision space is small and the honest answer is usually one of three codecs.

Opus: the best lossy codec, where you can use it

Opus is the current state of the art for lossy audio, and it is not close. It is royalty-free, standardised by the IETF, and it holds up across an unusually wide range: intelligible speech at 16 kbps, transparent music around 128 kbps stereo, and everything between. It also has very low algorithmic delay, which is why WebRTC mandates it and every video call you make already uses it.

The limitation is container and platform reach. Opus lives happily in WebM, Ogg and MP4, and every modern browser decodes it, but it is not the safe choice for a file that has to play on a smart TV or an older device. Inside a WebM file it is the obvious pairing, which is why our convert to WebM page defaults to it.

AAC: the compatibility default

AAC is what plays everywhere. Every phone, every TV, every browser, every editing tool. It is meaningfully better than MP3 at the same bitrate and meaningfully worse than Opus, and it is patent-encumbered in ways that matter to distributors more than to individuals.

For an MP4 destined for the open web, AAC is the correct choice for the same reason H.264 often is: not because it is the best, but because “it plays” beats “it is 15 percent smaller”.

Encoder choice matters more than people expect. FFmpeg’s native AAC encoder spent years behind Fraunhofer’s libfdk_aac and Apple’s, which is why the folklore says to avoid it. That advice is now out of date: the native encoder was rewritten with a new psychoacoustic model and, on FFmpeg’s own metrics and listening tests, now matches or beats both. The rewrite missed 9.0 and ships in 9.1.

MP3: legacy, and that is fine

MP3 is worse than AAC at every bitrate and its patents expired in 2017, making it freely usable. There is no technical reason to choose it for new work, and exactly one practical reason that still holds: some old hardware and some podcast tooling expects it. Use it when something downstream demands it, not otherwise.

Lossless: FLAC and ALAC

FLAC compresses audio with no loss at roughly half the size of WAV, is royalty-free, and is the default answer for archives and masters. ALAC is Apple’s equivalent, comparable in efficiency, and worth choosing only inside an Apple workflow.

Neither belongs in a delivery pipeline for video. Lossless audio alongside a lossy video track is a bitrate budget spent in the wrong place: the same bandwidth put into the video would be far more visible than the audio difference is audible.

Bitrates that are actually enough

Per stereo pair, for delivery:

• Speech only: Opus 24 to 32 kbps, or AAC 64 kbps. Podcasts routinely waste several times this.
• General video soundtrack: AAC 128 kbps, or Opus 96 kbps. This is the workhorse setting and the default on our extract audio page.
• Music you want to sound right: AAC 192 to 256 kbps, or Opus 128 to 160 kbps. Above this, the returns are inaudible to nearly everyone in nearly every listening environment.
• 5.1 surround: budget roughly 320 to 384 kbps in AAC, or 256 in Opus.

Two practical notes. Channel count multiplies the requirement, so a 5.1 mix at “128 kbps” is starved rather than efficient. And joint stereo is on by default in every modern encoder, which is why stereo does not cost twice mono.

The parts that break things

Sample rate. 48 kHz is the video standard; 44.1 kHz is the CD-era audio standard. Mixing them causes resampling somewhere in the chain, and a badly implemented resample is audible. Pick 48 kHz for anything paired with video.

Encoder delay. AAC encoders introduce a fixed priming delay, which is why AAC-in-MP4 carries edit lists to compensate. Tools that ignore those edit lists produce a few milliseconds of A/V drift, which is imperceptible alone and cumulative when files are concatenated.

Loudness, not peak. Every major platform normalises to a loudness target (around -14 LUFS for streaming music, -16 to -23 for broadcast). Mastering hot to 0 dBFS does not make you louder on those platforms; it makes you turned down with less dynamic range left. FFmpeg’s loudnorm filter measures and corrects to a target properly, in two passes.