How Automated Statistical Analysis Works
From data profiling through model search and scoring to generated explanation.
From data profiling through model search and scoring to generated explanation.
Types, ranges, missingness, cardinality, obvious duplicates, and the shape of the index. This step is cheap and catches a surprising share of real problems before any modelling starts.
Candidate structures are drawn from a grammar. In the Gaussian process formulation the base kernels are smooth (SE), periodic (PER), linear (LIN), changepoint (CP) and noise (WN), combined by addition and multiplication.
Each candidate is fitted and scored by marginal likelihood or an information criterion. Because a more flexible model spreads its probability mass more thinly, complexity is penalised automatically — see Bayesian Occam’s razor.
Simulate from the fitted model and compare against the observations. Where they diverge systematically, the model is describing the data badly regardless of its score relative to the alternatives it happened to be compared with.
Each component maps to a phrase. The product of a periodic and a smooth term becomes "a cycle whose amplitude varies smoothly"; a changepoint becomes "behaviour changes around 2008". Because the sentence is generated from the structure, it can be verified against it.
A grammar that cannot express the phenomenon will still return a winner. Search is greedy and can miss better structures behind a worse intermediate step. And nothing in the pipeline knows whether the question was worth asking.