A method for generating a labeled set of images for use in machine learning based stray light characterization for space-related optical systems. The method includes (a) obtaining a set of images simulated for a space-related optical system, wherein the images of the set of images contain stray light simulated for the space-related optical system, (b) for each image of the set of images, identifying one or more clusters of light contained in the respective image and labeling the respective image by the one or more clusters of light, wherein the one or more clusters of light include at least one cluster of stray light, and (c) creating, based on the labeled images of the set of images, a plurality of new labeled images by applying transformations to the labeled images to generate an augmented set of labeled images.