A human-computer interaction method based on neural signals relative to the visual movement relates to the technical filed of the human-computer interaction, and is characterized in that the method comprises the following steps: connecting a human-computer interaction unit waiting to be chosen with motional elements started at different times; arranging an EEG electrode in visual movement area ofthe user head for recording EEG signals; separately extracting brain waveforms corresponding to each motional element in a sequence according to the difference of the time starting; detecting whetheran N2 electric potential corresponding with the visual movement; detecting that a human-computer interaction unit corresponding with the simulation of the N2 electric potential is considered as a target for a tested person; finishing the corresponding functional module by a computer, so as to realize the human-computer interaction. The testing results by the system shows that the choosing of a target averagely costs 6 seconds, and both the detection rate and the false reject rate meet the requirement of the regular human-computer interaction. The method has the advantages of having no scintillations, hardly causing the visual fatigue, and being suitable for low contrast and low illumination condition, and different human-computer sites.