clm_a2c94d5dSpecifically, our experiments establish that state-of-the-art convolutional networks for image classification trained with stochastic gradient methods easily fit a random labeling of the training data.
arXiv 1611.03530 · cs.LG
Read on arXiv · arXiv non-exclusive license
Claims were extracted automatically by reducer-1 on 2026-11-12. The paper's license does not allow republishing it, so long claims are shortened to a quotation. Read the full text on arXiv.
clm_a2c94d5dSpecifically, our experiments establish that state-of-the-art convolutional networks for image classification trained with stochastic gradient methods easily fit a random labeling of the training data.
res_070bfb28More precisely, when trained on a completely random labeling of the true data, neural networks achieve 0 training error.
clm_7139efd5We establish this fact for several different standard architectures trained on the CIFAR10 and ImageNet classification benchmarks.
clm_b1d5a3f2We discuss in further detail below how these observations rule out all of VC-dimension, Rademacher complexity, and uniform stability as possible explanations for the generalization performance of state-of-the-art neural networks.
clm_f59bd4c3We show that explicit forms of regularization, such as weight decay, dropout, and data augmentation, do not adequately explain the generalization error of neural networks.
clm_43278963Our results show that even empirically training neural networks is not uniformly stable for many passes over the data.
res_5892e713Specifically, the top-1 accuracy without regularization is 59.80%, while random guessing only achieves 0.1% top-1 accuracy on ImageNet.
res_3f19a95eMore strikingly, with data-augmentation on but other explicit regularizers off, Inception is able to achieve a top-1 accuracy of 72.95%.
res_a9637192The normalization operator helps stablize the learning dynamics, but the impact on the generalization performance is only 3∼4%.
clm_e992508bWe argue that what is more relevant in practice is the expressive power of neural networks on a finite sample of size n.
res_cd4e2163On MNIST with no preprocessing, we are able to achieve a test error of 1.2% by simply solving (3).
res_ba174b91Simply applying a Gaussian kernel on pixels and using no regularization achieves 46% test error.
clm_08472d14We argue that we have yet to discover a precise formal measure under which these enormous models are simple.
Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (1611.03530) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.