A method for processing a stereo image is provided which includes the steps of setting a specified area of a base window with reference to a point for obtaining a corresponding point in the base image; setting a reference window having the same size as that of the base window in the reference image; evaluating the difference between the base window and the reference window while scanning the reference window along an epipolar line; determining the position of the reference window having the minimum difference as a stereo corresponding point; measuring parallax from the difference between the position of the base window and the position of the reference window; calculating the reliability of the corresponding point (parallax) by first reliability based on the sharpness of the peak of the evaluation-value distribution of the difference of patterns between the base window and the reference window and second reliability based on the shape of the peaks; and outputting the value of the reliability of the parallax together with information on the parallax.