An apparatus for displaying a stereoscopic image including right-eye and left-eye image display devices displaying right-eye and left-eye images, respectively, right-eye and left-eye magnifying lenses forming enlarged virtual right-eye and left-eye images, respectively, an eye point camera for detecting a point in a sight at which a viewer gazes, a parallax calculating circuit for deriving a parallax of the gazed point, a image shift amount calculating circuit for calculating an amount of image shift in accordance with the parallax, and an image shift circuits for shifting the right-eye and left-eye images horizontally in opposite direction by the same amount such that a distance of convergence with respect to the gazing point is substantially fixed. It is preferable to shift the right-eye and left-eye images such that the distance of convergence becomes substantially equal to a viewing distance from eyes of the viewer to a virtual image plane.