Video compression is one of the most sophisticated areas of computer science, built on exploiting two fundamental types of redundancy in video data: spatial redundancy and temporal redundancy. Understanding these concepts explains why modern codecs can achieve dramatic file size reductions while maintaining impressive visual quality.

Spatial redundancy exists within individual frames. Large areas of a single frame often contain similar colors and patterns, such as a blue sky, a white wall, or a stretch of road. Instead of storing the exact color value for every pixel, codecs divide each frame into blocks (typically 4x4 to 64x64 pixels) and apply mathematical transforms like the Discrete Cosine Transform (DCT) to convert spatial pixel data into frequency coefficients. High-frequency details (fine textures and sharp edges) can be aggressively quantized (reduced in precision) with minimal visual impact because the human eye is less sensitive to high-frequency information. This quantization step is where the actual lossy compression occurs and is the primary lever controlling the quality-versus-size tradeoff.

Temporal redundancy exists between consecutive frames. In most video content, the vast majority of each frame is identical or nearly identical to the previous frame. A talking head video might have 95% of pixels unchanged between frames. Codecs exploit this by storing only the differences (residuals) between frames using motion estimation and compensation. The encoder searches for blocks in reference frames that closely match blocks in the current frame, records a motion vector describing the displacement, and then encodes only the small residual difference. This is why I-frames (which cannot use temporal prediction) are dramatically larger than P-frames and B-frames.

Rate-distortion optimization (RDO) is the mathematical framework that guides encoding decisions. The encoder constantly evaluates the tradeoff between the number of bits spent (rate) and the visual error introduced (distortion) for every coding decision, including block size, prediction mode, quantization level, and motion vector precision. The goal is to minimize distortion at a given bit budget, or equivalently, minimize bits for a given quality target.

Two primary bitrate control strategies exist: Constant Bitrate (CBR) and Variable Bitrate (VBR). CBR maintains a fixed data rate throughout the video, which is required for streaming and broadcast applications where bandwidth is constrained. However, CBR wastes bits on simple scenes and starves complex scenes. VBR allows the encoder to allocate more bits to complex scenes with lots of motion or detail and fewer bits to simple, static scenes, resulting in more consistent visual quality throughout the video. Most modern compression tools use VBR or its refinement, CRF (Constant Rate Factor), which targets a consistent perceptual quality level rather than a specific bitrate.