Implementing a neural network includes determining whether to process a combination of a first region of an input feature map and a first region of a convolution kernel and, responsive to determining to process the combination, performing a convolution operation on the first region of the input feature map using the first region of the convolution kernel to generate at least a portion of an output feature map.