A line-of-sight detecting device includes an area sensor composed of a plurality of photoelectric conversion elements for receiving an image of an eyeball of a viewer. The area sensor is divided into block areas each having some of the plurality of photoelectric conversion elements, each block area outputs a value on the basis of the output of the photoelectric conversion elements constituting that block area. A particular block area satisfying predetermined conditions based on the output values of the block areas is selected from the plurality of block areas. Image processing is performed on those photoelectric conversion elements which are in a predetermined range based on the position of the particular block area to thereby detect the line of sight of the viewer, wherein the number of photoelectric conversion elements constituting each block area is changed in accordance with a predetermined situation.