A digital video computing system receives two or more frames depicting an environment from a camera system of a vehicle. For a salient image feature identified in the two or more frames, a global motion vector is calculated that is indicative of movement of the feature at least partially attributable to movement of the vehicle. A local motion vector is calculated that is indicative of movement of the feature independent from the movement of the vehicle. Based on the local motion vector, the salient image feature is determined to have an apparent motion relative to the environment that is independent from the movement of the vehicle. A candidate image patch is identified including the salient image feature. The candidate image patch is analyzed to output a likelihood that the candidate image patch depicts a second vehicle.