A stereo matching apparatus and method through learning a unary confidence and a pairwise confidence are provided. The stereo matching method may include learning a pairwise confidence representing a relationship between a current pixel and a neighboring pixel, determining a cost function of stereo matching based on the pairwise confidence, and performing stereo matching between a left image and a right image at a minimum cost using the cost function.