The present disclosure relates to an image merging system that automatically and seamlessly detects and merges missing people for a set of digital images into a composite group photo. For instance, the image merging system utilizes a number of models and operations to automatically analyze multiple digital images to identify a missing person from a base image, segment the missing person from the second image, and generate a composite group photo by merging the segmented image of the missing person into the base image. In this manner, the image merging system automatically creates merged group photos that appear natural and realistic.