Systems and methods for augmenting a data set are provided. An example method may include locating a foreground object disposed within a seed image, identifying an object class corresponding to the foreground object, and, based on the identified object class, determining a target value for an object property of the foreground object. The example method may also include applying a transformation function to transform the foreground object into a transformed object, where the transformation function modifies the object property of the foreground object from having an initial value to having the target value. The example method may further include transplanting the transformed object into a background image so as to produce an augmented image and augmenting an initial set of images with the augmented image so as to produce an augmented set of images for training a predictive model.