Systems and methods for selecting representative photos for a photo album are described. An example computing device may include a display, a logic device, and a storage device storing instructions executable by the logic device to, for each known individual recognized in one or more photos of the album, determine an affinity score for that known individual indicating a connection of that known individual to a) a user of the computing device and b) a context corresponding to the album. The instructions may be further executable to select one or more known individuals based on the affinity score to be included in a subset of the known individuals recognized in one or more photos of the album, and select a representative photo of each of the subset of the known individuals to be included in one or more of a highlight photoset and a cover photo for the album.