What Is Automated Exploratory Data Analysis?
What EDA profilers compute, what they miss, and how to read their output without being misled.
What EDA profilers compute, what they miss, and how to read their output without being misled.
Per column: type, count, distinct values, missing count, min/max, quantiles, mean and standard deviation, a histogram, and often a flag for skew or high cardinality. Across columns: a correlation matrix and sometimes a duplicate-row check.
Whether a zero means zero or means missing. Whether two columns are the same measurement recorded twice. Whether the sample is representative. Whether a spike is an event or an import error. These are the questions that decide the analysis.
A profile over 40 columns produces 780 pairwise correlations. At a 5% threshold you should expect around 39 to look interesting by chance alone. Automated EDA output invites exactly this mistake.
The count is less informative than the pattern. Values missing together across rows usually indicate a broken join or a form field that appeared later. Values missing in a run indicate an outage. Values missing conditional on another column indicate something structural, and dropping those rows will bias the result.
Treat every flag as a question for someone who knows how the data was collected. The profile is a fast way to generate that list, which is genuinely valuable — just not the same thing as an analysis.