Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a convolutional neural network using a regularization scheme. One of the methods includes repeatedly performing the following operations: obtaining a kernel of a particular convolutional layer; applying a Fourier transform to the kernel; generating a decomposition using singular-value decomposition (SVD); generating a regularized diagonal matrix; generating a recomposition; applying an inverse Fourier transform to the recomposition; and training the convolutional neural network on training inputs.