Why a small training loss is not the same as having learned the function, how bias and variance split the blame, and why interpolating models are not automatically useless. Fill every blank by clicking. Nothing here is scored. Dotted terms carry an i — open it.
References: C. M. Bishop, Pattern Recognition and Machine Learning (2006), §§1.1 and 3.2;
I. Goodfellow, Y. Bengio & A. Courville, Deep Learning (2016), §§5.2–5.4 and ch. 7;
C. M. Bishop & H. Bishop, Deep Learning: Foundations and Concepts (2023), §§4.3 and 9.3.2;
S. J. D. Prince, Understanding Deep Learning (2023/25), ch. 20.
Double descent: Belkin et al. (2019); Nakkiran et al. (2020).