Provided are a neuromorphic computing device, memory device, system, and method to maintain a spike history for neurons in a spiking neural network. A neural network spike history is generated in a memory device having an array of rows and columns of memory cells. There is one row of the rows for each of a plurality of neurons and columns for each of a plurality of time slots. Indication is made in a current column in the row of the memory cells for a firing neuron that a spike was fired. Indication is made in the current column in rows of memory cells of idle neurons that did not fire that a spike was not fired. Information in the array is used to determine a timing difference between a connected neuron and the firing neuron and to adjust a weight of the connecting synapse.