Regularisation · Practice sheet · L2 → L1 → Dropout → BN → Early Stopping
0 / 0 blanks filled

all sheets

deep learning · regularisation

Five ways to spend capacity more slowly

L2 shrinks every weight, L1 deletes some of them, dropout deletes activations for a step, batch normalisation re-scales a minibatch, and early stopping deletes training time. Fill every blank by clicking. Nothing here is scored. Dotted terms carry an i — open it. Textbook figures are screenshots of the cited pages.

References: C. M. Bishop, Pattern Recognition and Machine Learning (2006), §§1.1, 3.1.4, 5.5.2; I. Goodfellow, Y. Bengio & A. Courville, Deep Learning (2016), ch. 7 and §8.7.1; C. M. Bishop & H. Bishop, Deep Learning: Foundations and Concepts (2023), §§7.4.2 and ch. 9; S. J. D. Prince, Understanding Deep Learning (2023/25), ch. 9 and §11.4. Dropout: Srivastava et al. (2014). Batch normalisation: Ioffe & Szegedy (2015).