The invention discloses a brain-computer interface system and a control method thereof, belonging to the technical field of BCIs (brain-computer interfaces). The brain-computer interface system comprises an EEG (electroencephalogram) data acquisition module, a first data processing module, a second data processing module and a pattern recognition module, wherein the EEG data acquisition module isused for displaying each level of a Speller stimulating interface on the screen of the EEG data acquisition module according to a first preset time interval and acquiring the EEG signals of users; the first data processing module is used for processing the EEG signals, outputting control signals and selecting the level in which a character to be selected is located; the second data processing module is used for processing the EEG signals,and obtaining visual evoked P300 signals; and the pattern recognition module is used for carrying out classification and identification on the visual evoked P300 signals, and then outputting the character to be selected. By using the system and method disclosed by the invention, the number of the character to be selected is increased, and the information transmission efficiency is effectively improved, thereby ensuring considerable social benefits and economic benefits.