A focus detection system includes an object lens for focusing an image of an object, a refocusing optical system for refocusing a plurality of component image having parallax from the image of the image, a view field mask arranged between the object lens and the refocusing optical system and having a laterally extending aperture, and a sensing device having a photo-sensor array for sensing the component images. Focus detection error is prevented by determining a sense area of the photo-sensor array in accordance with a distortion created in an aperture image by the refocusing optical system.