A stereoscopic imaging apparatus which uses contrast AF to obviate disadvantages of phase-difference AF, shorten processing time of contrast AF, and improve focusing accuracy. A phase-difference CCD in which object images having passed through different regions in a predetermined direction of a single photographing optical system are pupil-split and imaged subjects the object images having passed through the respective regions to photoelectric conversion to obtain a main pixel image and a sub-pixel image which have a mutual phase difference. Contrast AF is performed using only one of the main pixel image and the sub-pixel image. Which one of the main pixel image and the sub-pixel image will be used for the contrast AF is determined depending on whether the position of an AF area in the angle of view belongs to a main pixel AF area or a sub-pixel AF area.