Gaussian Process Kernels Explained
What each kernel assumes, what it produces, and how combinations change the story.
What each kernel assumes, what it produces, and how combinations change the story.
A kernel maps a pair of inputs to a covariance. Large covariance at large separation means the function is smooth and slow; covariance that decays quickly means it is rough. The length-scale sets the distance over which the function is expected to change appreciably.
Squared exponential (SE) — smooth to infinite order; good for trends, over-smooth for physical processes. Matérn — a roughness parameter makes it more realistic for sensor and physical data. Periodic (PER) — exact repetition at a fixed period. Linear (LIN) — straight-line behaviour, useful multiplied rather than alone. White noise (WN) — independent observation error. Changepoint (CP) — a smooth switch between two kernels at a location.
| Expression | Reads as |
|---|---|
| SE + PER | A smooth trend plus a stable cycle |
| SE × PER | A cycle whose amplitude varies smoothly |
| LIN × PER | A cycle whose amplitude grows linearly |
| CP(t) × SE | Smooth behaviour that changes at time t |
| SE + WN | A smooth signal observed with noise |
Hand-choosing works when the phenomenon is understood — a daily cycle in web traffic, an annual cycle in retail. Searching pays off when it is not, and when the interaction terms are where the story is, as in the airline series.