An apparatus and a method for processing a stereo image are provided. In the method, a first image and a second image are obtained via a first camera module and a second camera module installed spaced by a predetermined distance. When the stereo image is captured, the first image and the second image are alternatively sampled with a predetermined time interval. The sampled at least one first image and the sampled at least one second image are encoded. Therefore, the stereo image can be compressed with low complexity compared to the case of compressing a stereo image using a 2D image compression technology.