Depth information can be used to assist with image processing functionality, such as various calibration approaches to determine and correct for misalignments between cameras. In at least some embodiments, depth information obtained from stereo imaging or distance sensing, for example, can be used to segment an image or frame of video into at least a foreground object and a background object. Once the foreground and background object has been determined, information about that objects can be used to determine a mapping, and once a subsequent stereoscopic image is captured using the cameras, the mapping can be applied to each image to account for misalignment effects due to a misaligned stereo camera pair before providing the stereoscopic image for display.