Video rotation can be achieved through two fundamentally different approaches: metadata-based rotation and pixel-level rotation. Understanding the distinction is important because they produce very different results in terms of processing speed, file size, compatibility, and quality preservation.

Metadata-based rotation is the approach used by modern smartphones. When you hold your phone in landscape while recording, the camera sensor always captures pixels in the same physical orientation, but the phone writes a rotation flag into the video file metadata that tells players how to display the frames. In the MP4 container format, this is stored in a structure called the display matrix (also known as the transformation matrix) within the track header (tkhdr) atom. This 3x3 affine transformation matrix can represent rotation, scaling, and translation operations. For a 90-degree clockwise rotation, the matrix contains specific trigonometric values that instruct the player to rotate the frame before displaying it. The QuickTime MOV format uses an identical mechanism, while MKV and WebM store rotation as a simple angle value in their track metadata.

The advantage of metadata rotation is that it requires no re-encoding. The actual pixel data remains untouched, and only a few bytes of metadata are modified, making the operation instantaneous regardless of video length or resolution. The disadvantage is compatibility: not all video players correctly interpret rotation metadata. Older software, some web browsers, and many video editing tools may ignore the rotation flag entirely, displaying the video in its native sensor orientation. This is why many users encounter the problem of a video appearing correctly on their phone but sideways when transferred to a computer or uploaded to certain platforms.

Pixel-level rotation physically rearranges the pixel data in every frame of the video. For a 90-degree rotation, each frame's pixel grid is transposed and mirrored: a pixel at position (x, y) in the original frame moves to position (height - y - 1, x) in the rotated frame. This means the width and height dimensions of the video swap, changing a 1920x1080 video to 1080x1920. Because the pixel data is fundamentally restructured, this approach requires full decoding and re-encoding of the video stream, which is computationally expensive and introduces generation loss from the re-encoding step. However, the result is universally compatible since any player will display the correct orientation without needing to interpret metadata.

For 180-degree rotation and horizontal or vertical flipping, the pixel rearrangement is simpler. A 180-degree rotation reverses both the row and column order of pixels. A horizontal flip (mirror) reverses only the column order within each row, while a vertical flip reverses the row order. Some container formats and codecs support signaling these simpler transformations through metadata as well, using orientation flags similar to EXIF orientation tags in JPEG images. The choice between metadata and pixel-level approaches depends on the specific use case, with metadata preferred for speed and quality when the destination supports it, and pixel-level rotation used when universal compatibility is required.