A method and system for training a neural network are described. The method includes providing at least one continuously differentiable model of the neural network. The at least one continuously differentiable model is specific to hardware of the neural network. The method also includes iteratively training the neural network using the at least one continuously differentiable model to provide at least one output for the neural network. Each iteration uses at least one output of a previous iteration and a current continuously differentiable model of the at least one continuously differentiable model.