A method and apparatus for detecting moving objects in both two-dimensional and three-dimensional scenes. The method repetitively applies a two-dimensional transformation to a plurality of images representing a scene to identify misaligned regions within the images. Any residual motion represented by the misaligned regions that may be classified as a moving object within the scene is further processed by a three-dimensional technique that removes parallax motion from the residual motion. The result is motion contained in an epipolar flow field which is only due to a moving object within the scene.