Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting a neural network architecture for performing a machine learning task. In one aspect, a method comprises: obtaining data defining a synaptic connectivity graph representing synaptic connectivity between neurons in a brain of a biological organism; generating data defining a plurality of candidate graphs based on the synaptic connectivity graph; determining, for each candidate graph, a performance measure on a machine learning task of a neural network having a neural network architecture that is specified by the candidate graph; and selecting a final neural network architecture for performing the machine learning task based on the performance measures.