Text extraction from PDF documents is one of the most deceptively challenging tasks in document processing, because PDF was designed as a final-form presentation format — optimized for precise visual rendering rather than for content reuse. Unlike HTML or word processor formats where text flows in a logical reading order with explicit paragraph and sentence structure, PDF positions individual characters and character groups at exact coordinates on the page with no inherent concept of words, sentences, or paragraphs.

The PDF text rendering model works through text operators in content streams. The key operators are Tj (show a string), TJ (show strings with individual glyph positioning), Tm (set the text matrix for positioning), and Td/TD (move to the next line). A typical content stream positions text by setting a transformation matrix, then outputs a string of character codes. Critically, these character codes are not necessarily Unicode — they are indices into the font's encoding, which may be a standard encoding like WinAnsiEncoding, a custom encoding defined in the font dictionary, or an identity encoding for CID fonts used with East Asian character sets.

The ToUnicode CMap is the primary mechanism for translating these internal character codes back to readable Unicode text. When a PDF producer embeds a font, it should include a ToUnicode mapping that defines the correspondence between each character code used in the document and its Unicode code point. Without this mapping, an extractor must fall back to the font's built-in encoding or Differences array, and in the worst case — when fonts use custom encodings without any Unicode mapping — character codes cannot be reliably converted to text, resulting in garbled output.

Reconstructing words and paragraphs from individually positioned characters requires spatial analysis. The extractor must examine the positioning of each glyph and determine whether the gap between consecutive characters represents a normal inter-character space, a word boundary, or a column/paragraph break. This involves comparing the distance between glyphs against the font's expected character width and space width. Characters on the same baseline with small gaps are grouped into words; larger gaps indicate word boundaries; vertical displacement indicates line breaks. Determining paragraph breaks versus line breaks within a paragraph requires heuristics based on indentation, line spacing, and alignment patterns.

Multi-column layouts present additional challenges because text from different columns may be interleaved in the content stream. A document with two columns might render the first line of column one, then the first line of column two, alternating throughout the page. The extractor must detect column boundaries through spatial analysis and reassemble text in logical reading order. Tables are even more complex, as cell text must be associated with the correct row and column based on position. Right-to-left text, vertical text (common in CJK documents), and mixed-direction text add further complexity. Ligatures — where multiple characters are represented by a single glyph — must be decomposed back into their component characters using the ToUnicode map. All of these challenges explain why even the best PDF text extractors sometimes produce imperfect results, particularly with complex layouts.