Hardware optimization of neural networks is provided. In various embodiments, an output-induced receptive field of each of a plurality of layers of a neural network is determined. From each of the plurality of layers any portions of their respective input that falls outside their respective output-induced receptive field are trimmed. For each of the plurality of layers, a plurality of mappings of the layer to physical neurosynaptic cores are determined. A mapping is determined having a minimum total number of cores required for the neural network based on the plurality of mappings.