A controller for an autonomous vehicle receives an image stream from one or more imaging devices. The controller identifies vehicle images in the image stream. Vehicle images are compared to the color, shape, badging, markings, license plate, and driver of the autonomous vehicle. If the vehicle image is determined to match the autonomous vehicle, then the vehicle image is ignored as a potential obstacle. The location of a reflective surface that generated the vehicle image may be determined and added to a set of potential obstacles. The color and shape of a vehicle in a vehicle image may be evaluated first. Only if the color and shape in the vehicle image match the autonomous vehicle are other factors such as badging, markings, license plate, and driver considered. Vehicle images not matching the autonomous vehicle are included in a set of potential obstacles.