A processor-implemented speech recognition method includes: applying, to an input layer of a neural network, a frame of a speech sequence; obtaining an output of a hidden layer of the neural network corresponding to the frame; calculating a statistical value of at least one previous output of the hidden layer corresponding to at least one previous frame of the speech sequence; normalizing the output based on the statistical value; applying the normalized output to a subsequent layer of the neural network; and recognizing the speech sequence based on the applying of the normalized output.