An architecture search system evaluates a search space of neural network and hardware architectures with a plurality of candidate controllers. Each controller attempts to identify an optimized architecture using a different optimization algorithm. To identify a controller for the search space, the architecture search system samples subspaces of the search space having a portion of the neural network search space and a portion of the hardware search space. For each subspace, candidate controllers are scored with respect to the optimized design determined by the respective candidate controllers. Using the scores for the various candidate controllers across the sampled subspaces, a controller is selected to optimize the overall network architecture search space.