Missing data analysis is a cornerstone of data quality assessment, rooted in statistical theory developed by Donald Rubin and Roderick Little in the 1970s and 1980s. Their taxonomy of missing data mechanisms—Missing Completely At Random (MCAR), Missing At Random (MAR), and Missing Not At Random (MNAR)—provides the theoretical framework for understanding why data is absent and what consequences that absence has for analysis. The mechanism of missingness determines which analytical techniques remain valid and which produce biased results.

Missing Completely At Random (MCAR) means the probability of a value being missing is unrelated to both the missing value itself and all other observed values. This is the most benign form of missingness—like a survey response lost in the mail. MCAR data can be analyzed with complete-case analysis (simply excluding incomplete records) without introducing bias, though statistical power is reduced. Missing At Random (MAR) means the missingness depends on observed values but not the missing values themselves—for example, younger respondents being less likely to report income, regardless of their actual income level. MAR data requires more sophisticated handling like multiple imputation. Missing Not At Random (MNAR) is the most problematic: the probability of missingness depends on the missing value itself—high-income individuals refusing to report income specifically because it is high. MNAR data requires modeling the missing data mechanism explicitly.

Practical missing data detection must identify multiple representations of absence. Empty strings, the most obvious form, represent fields with no content. However, data systems use numerous conventions to represent missing values: NULL in databases, NA and N/A in statistical software, NaN (Not a Number) for undefined computations, "none," "missing," dash or hyphen characters, and even specific sentinel values like 9999 or -1. Comprehensive detection requires checking against all common representations to avoid underestimating the true level of missing data.

Column-level completeness metrics provide an immediate data quality overview. A column with 99% completeness has minimal missing data and likely supports reliable analysis. A column with 50% completeness requires careful consideration—is the missing data informative, or does it make the column unsuitable for analysis? Comparing completeness across columns reveals patterns: if multiple columns have similar completeness percentages, their missing values may overlap in the same rows, suggesting systematic issues like incomplete record entry.

Row-level analysis complements column-level metrics by identifying specific records with missing values. Records missing a single field may be usable for most analyses, while records missing many fields may need to be excluded entirely. Identifying rows with the most missing values often reveals data entry issues, import failures, or systematic problems with specific data sources. This granular analysis enables targeted data cleaning efforts focused on the most impactful records and fields, maximizing data quality improvement per unit of effort.