A method for reducing the number of neurons in a trained deep neural network (DNN) includes classifying layer types in a plurality of hidden layers; evaluating the accuracy of the DNN using a validation set of data; and generating a layer specific ranking of neurons, wherein the generating includes: analyzing, using the validation set of data for one or more of the plurality of hidden layers, the activation function for each neuron in the analyzed layers to determine an activation score for each neuron; and ranking, on a layer type basis, each neuron in the analyzed layers based on the neuron's activation score to generate a layer specific ranking of neurons. The method further includes removing a number of lower ranked neurons from the DNN that does not result in the DNN after the removal of selected lower ranked neurons to fall outside of an accuracy threshold limit.