A phase difference detector of the type capable of distinguishing a focused state by detecting relative positions of a pair of optical images obtained from an object, comprises sensor means adapted to photoelectrically convert the pair of optical images and to output a first analog electrical signal corresponding to one of the optical images and a second analog electrical signal corresponding to the other optical image, the signals being generated by the photoelectric conversion, at a predetermined period and in a non-destructive manner, and analog arithmetic means for subjecting the pair of analog electrical signals output from the sensor means to a correlative arithmetic operation.