A video signal processing device performs signal processing on a baseband video signal decoded from a compressed video signal, and includes: an interpolation phase generating unit which obtains frame repeat information indicating positions of identical frames output sequentially, and generates interpolation phase information for generating an interpolation frame in which a motion between frames is interpolated; a motion vector estimating unit which estimates a motion vector between mutually different frames indicated by the frame repeat information; and an interpolation frame generating unit which generates the interpolation frame using the interpolation phase information and the motion vector, and outputs an output video signal having a frame rate equal to the frame rate of the baseband video signal.