The optical flow in an image is estimated using only parameters extracted directly from a compressed video data stream, substantially eliminating the need to decode the video data stream for this purpose. Coefficients extracted from the compressed video data stream are used to establish a confidence map indicative of the edge strength within the image data and hence the accuracy of the associated motion field. A smooth motion field is generated from the motion vectors inherent in the compressed video data stream. The motion field is then used to update the confidence map between frames, providing an estimate of the optical flow.