In embodiments of optical flow with nearest neighbor field fusion, an initial motion field can be generated based on the apparent motion of objects between digital images, and the initial motion field accounts for small displacements of the object motion. Matching patches of a nearest neighbor field can also be determined for the digital images, where patches of an initial size are compared to determine the matching patches, and the nearest neighbor field accounts for large displacements of the object motion. Additionally, region patch matches can be compared and determined between the digital images, where the region patches are larger than the initial size matching patches. Optimal pixel assignments can then be determined for a fused image representation of the digital images, where the optimal pixel assignments are determined from the initial motion field, the matching patches, and the region patch matches.