A motion vector detecting apparatus includes an evaluation value information forming unit, a motion vector extracting unit, and a motion vector deciding unit. The evaluation value information forming unit is configured to form evaluation value information on motion vectors evaluating that a reference pixel is a potential movement destination candidate for a focus pixel. The evaluation value information on the motion vectors is generated from pixel value correlation information that has been one of generated and limited using a movement direction of the pixels. The motion vector extracting unit is configured to extract motion vectors as candidates from a plurality of motion vectors based on the evaluation value information formed by the evaluation value information forming unit. The motion vector deciding unit is configured to decide motion vectors to be assigned to pixels in a frame from the motion vectors extracted as the candidates by the motion vector extracting unit.