A stereoscopic video processing apparatus includes: a switch operation detection unit which detects a switch operation, made by a user, that is an operation to select a channel of a three-dimensional (3D) video or an operation for a fast-forward playback, a rewind playback, skipping, or locating a cue point of a 3D video; a frequency determination unit which determines a detection frequency indicating the number of switch operations per unit of time detected by the switch operation detection unit; and a video processing unit which performs processing to reduce a difference in depth between 3D videos before and after switching of the 3D videos when the detection frequency per unit of time determined by the frequency determination unit is higher than or equal to a predetermined frequency.