The research themes behind automated, interpretable statistical modelling — model discovery, Gaussian processes, natural-language explanation and model criticism — together with the historical publications that established them.
Historical research is credited to its original authors and institutions. Commentary and modern context are written by this site’s editorial team.
The project began from an observation about scarcity: the volume of data being collected was growing much faster than the number of people able to analyse it properly. The response was not to build a better single model but to automate the process a statistician goes through — propose a structure, fit it, criticise it, revise it — and to keep the output legible at every step.
Four ingredients define the approach. An open-ended language of models, expressive enough to describe real phenomena. A search procedure to explore that language efficiently. A principled method of evaluating candidates that trades complexity against fit. And a translation step that turns the winning structure into prose, figures and caveats a reader can act on. Each ingredient is a research problem in its own right, and each is where modern automated analysis still tends to fail.
If models can be composed from a small set of building blocks, the set of possible models becomes a grammar, and choosing a model becomes a search problem. That reframing is what makes automation tractable — and what keeps the winner interpretable, because the winning expression is built from parts that each carry a meaning.
A Gaussian process places a distribution over functions, with the covariance function carrying every assumption about shape. Because those assumptions are structured objects rather than free parameters, a discovered model decomposes into named components, each of which corresponds to a phrase in ordinary language: a smooth trend, a repeating cycle of fixed period, steady linear growth, behaviour that changes at a point in time, a cycle whose amplitude varies, and unexplained variation.
That is the property the whole approach rests on: the model does not merely predict, it says what kind of thing it thinks the data is. The Gaussian Processes hub covers the mechanics.
Once a model decomposes into named components, each component can be described. A periodic term becomes a sentence about a cycle; a product of a periodic and a linear term becomes a sentence about a cycle that grows. The description is generated from the structure itself, not from a language model paraphrasing a chart.
This distinction matters more now than it did when the research was published. Text generated from a fitted structure is checkable against that structure. Text generated from a prompt is not.
A search procedure always returns a winner. That is its job, and it is also its most dangerous property: the best model in a badly chosen space can still be a poor description of the data. Model criticism is the step that asks whether the winner is any good in absolute terms — comparing simulated data from the fitted model against the observations and reporting where they diverge.
An automated system that reports only its best model is providing half an answer. The half it omits is the half a statistician would have insisted on.
These papers and theses were produced by the original researchers at the University of Cambridge and MIT. They are listed here for reference and are not the work of this website’s editorial team.
| Paper | Authors | Venue |
|---|---|---|
| Structure Discovery in Nonparametric Regression through Compositional Kernel Search | Duvenaud, Lloyd, Grosse, Tenenbaum, Ghahramani | ICML 2013 |
| Automatic Construction and Natural-Language Description of Nonparametric Regression Models | Lloyd, Duvenaud, Grosse, Tenenbaum, Ghahramani | AAAI 2014 |
| Statistical Model Criticism using Kernel Two Sample Tests | Lloyd, Ghahramani | NIPS 2015 |
| Exploiting compositionality to explore a large space of model structures | Grosse, Salakhutdinov, Freeman, Tenenbaum | UAI 2012 |
Related student and doctoral work at the University of Cambridge includes Kernel Structure Discovery for Gaussian Process Classification (Nikola Mrkšić, 2014), Towards an Artificial Intelligence for Regression (Riaz Moola, 2015) and The Automatic Statistician for Classification (Qiurui Charles He, 2016).