A stereoscopic image generating method and device thereof. The stereoscopic image generating device comprises a storage unit for storing a plurality of first images, a plurality of second images, a plurality of disparity maps and a pixel density value, and a processing unit. The processing unit is configured to obtain viewing data including a pupil distance associated with a pupil of a viewer, a first constant value, and a second constant value. For each of the first image and each of the second images, after receiving the location information, an aggregate disparity value is obtained according to the disparity map corresponding to the first image and the second image and the location information; a conversion model based on at least the viewing data and the disparity value of the aggregate is generated; then a converted disparity map is formed according to the conversion model; finally, according to the converted disparity map, the first image and the second image are combined into astereoscopic image. The method and device is capable of instantly optimizing a stereoscopic image in accordance with the gaze position of the viewer.