Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for neural machine translation. In one aspect, a system is configured to receive an input sequence of source embeddings representing a source sequence of words in a source natural language and to generate an output sequence of target embeddings representing a target sequence of words that is a translation of the source sequence into a target natural language, the system comprising: a dilated convolutional neural network configured to process the input sequence of source embeddings to generate an encoded representation of the source sequence, and a masked dilated convolutional neural network configured to process the encoded representation of the source sequence to generate the output sequence of target embeddings.