A processor-implemented neural network operation method includes: receiving a first activation gradient and a first threshold corresponding to a layer included in a neural network; sparsifying the first activation gradient based on the first threshold; determining a second activation gradient by performing a neural network operation based on the sparsified first activation gradient; determining a second threshold by updating the first threshold based on the second activation gradient; and performing a neural network operation based on the second activation gradient and the second threshold.