A neural network element, outputting an output signal in response to a plurality of input signals, comprises a history memory for accumulating and storing the plurality of input signals in a temporal order as history values. It also includes an output module for outputting the output signal when an internal state exceeds a predetermined threshold value, the internal state being based on a sum of the product of a plurality of input signals and corresponding coupling coefficients. The history values depend on change of the internal state. The neural network element is configured to subtract a predetermined value from the internal state immediately after the output module fires and performs learning for reinforcing or attenuating the coupling coefficient according to the history values after the output module fires.