A conference data processing method includes that a conference terminal collects an audio segment in a first conference site based on a sound source direction in a conference process, generates first additional information corresponding to each of the collected audio segments, and sends, to a conference information processing device, a conference audio recorded in the conference process and the first additional information; the conference information processing device segments the conference audio into a plurality of audio segments and attaches corresponding second additional information to the plurality of audio segments, where the second additional information corresponding to each audio segment includes information used to determine a speaker identity corresponding to the audio segment and identification information of the corresponding audio segment, and the conference information processing device generates a correspondence between a participant and a statement based on the first additional information and the second additional information.