A distribution over functions, the kernel that encodes your assumptions, and an honest interval around every prediction.
Elizabeth Sramek
Published 21 July 2026
Most regression methods ask you to choose a functional form and then fit its parameters. A Gaussian process inverts the question. You specify what a plausible curve looks like — how smooth, how wiggly, whether it repeats — and let the data narrow the field. What comes back is a distribution over all curves consistent with both your assumptions and the observations.
A Gaussian process is a collection of random variables, any finite subset of which is jointly Gaussian. Pick any set of input points and the function values there follow a multivariate normal, described entirely by a mean function and a covariance function.
The mean is usually set to zero, which says only that before seeing data there is no reason to expect the function above or below zero. Everything interesting is in the kernel.
The kernel answers one question: how similar should function values at two inputs be, given how far apart the inputs are? Squared-exponential produces smooth curves; periodic produces repetition; linear produces straight lines.
| Kernel | Assumption | Typical use |
|---|---|---|
| SE | Smooth, infinitely differentiable | Slow-moving trends |
| PER | Repeats with fixed period | Seasonality, cycles |
| LIN | Linear in the input | Growth, drift |
| MAT | Rough, finitely differentiable | Physical and sensor data |
| WN | Independent observation error | Measurement noise |
A wide interval is not a failure. It is the model reporting that the data does not constrain the function in that region. Methods returning a single number in the same situation are not more certain — they are silent about it.
Adding kernels models a sum of independent components; multiplying them produces interaction, such as a periodic pattern whose amplitude grows over time. That closure property is the engine behind automated structure discovery.
from sklearn.gaussian_process.kernels import RBF, ExpSineSquared, WhiteKernel
trend = 1.0 * RBF(length_scale=60.0)
seasonal = 1.0 * ExpSineSquared(length_scale=1.5, periodicity=12.0)
noise = WhiteKernel(noise_level=0.1)
kernel = trend * seasonal + trend + noiseExact inference is O(n³) in time and O(n²) in memory, so past roughly ten thousand points you need sparse approximations or structured kernels. High-dimensional inputs degrade the distance notion the kernel relies on; for wide tabular data, gradient-boosted trees usually win and cost far less.
Both, and the distinction is not useful here. GPs are a Bayesian nonparametric method with a long statistical history that became standard in machine learning because they handle small data and uncertainty well.
Less than most methods — dozens or hundreds of points, where the prior does real work. The constraint is at the other end: exact inference becomes impractical past a few thousand observations.
A composed kernel decomposes into named parts, and each maps to a phrase such as “a smooth trend” or “a twelve-month cycle with growing amplitude”. Translating that decomposition into prose is what the original research demonstrated.