Why accuracy alone is a poor criterion when someone has to defend the result.
Elizabeth Sramek
Published 25 August 2026
Black-box AutoML searches ensembles and stacks for the best validation score. Explainable AutoML restricts the search to model families whose fitted form can be read — additive structures, monotone constraints, compositional kernels — accepting some performance cost.
| Dimension | Black-box AutoML | Explainable AutoML |
|---|---|---|
| Objective | Validation metric | Metric subject to interpretability constraints |
| Output | Score and a pipeline | Structure, components, uncertainty |
| Explanation | Post-hoc attribution | Read from the model itself |
| Best for | Prediction at scale | Decisions that must be justified |
Attribution methods fit a simple local surrogate to a complex surface. Where the surface is smooth the surrogate is fine; where it is not — precisely the regions where behaviour is surprising and the explanation would be most useful — the surrogate can be badly wrong, and nothing in the output signals it.
If a wrong answer costs someone money, a licence, or a job, the model has to be defensible in its own terms. If the cost of a wrong answer is a slightly worse recommendation, take the score.