Example embodiments relate to identifying defects in optical detector systems based on extent of stray light. An example embodiment includes a method. The method includes capturing, using an optical detector system, an image of a scene that includes a bright object. The method also includes determining a location of the bright object within the image. Further, the method includes determining, based on the location of the bright object within the image, an extent of stray light from the bright object that is represented in the image. In addition, the method includes determining, by comparing the extent of stray light from the bright object that is represented in the image to a predetermined threshold extent of stray light, whether one or more defects are present within the optical detector system. The predetermined threshold extent of stray light corresponds to an expected extent of stray light.