Systems and methods are provided for implementing hardware optimization for a hardware accelerator. The hardware accelerator emulates a neural network. Training of the neural network integrates a regularized pruning technique to systematically reduce a number of weights. A crossbar array included in hardware accelerator can be programmed to calculate node values of the pruned neural network to selectively reduce the number of weight column lines in the crossbar array. During deployment, the hardware accelerator can be programmed to power off periphery circuit elements that correspond to a pruned weight column line to optimize the hardware accelerator for power. Alternatively, before deployment, the hardware accelerator can be optimized for area by including a finite number of weight column line. Then, regularized pruning of the neural network selectively reduces the number of weights for consistency with the finite number of weight columns lines in the hardware accelerator.