Video data is processed with region-based warping. In accordance with various example embodiments, video data having a plurality of source video frames is processed as follows, for each source video frame. Pixels representing the source video frame and at least one temporally-adjacent video frame are used to generate a motion history map, based upon frame-to-frame movement of at least one region in the source video frame and temporally-adjacent video frames. At least one salient region is identified in the source video frame based upon the motion history map and static image saliency data. Weighting data is determined for warping pixels corresponding to background regions of the source video frame differently than pixels corresponding to the at least one salient region in the source video frame. Using the weighting data, the pixels representing the source video frame are warped to generate pixels representing a warped video frame.