One embodiment of the present invention provides a system that uses directional weighting functions while computing optical flow for an image through belief propagation. The system operates by computing local velocity estimates for points in the image. The system also computes directional weighting functions for points in the image, wherein the directional weighting functions facilitate propagating information in a manner which can vary with direction. The system subsequently uses these directional weighting functions during a belief propagation process, which propagates the local velocity estimates between points in the image, and which computes an optical flow field for points in the image.