Implementations relate to automatic sharing of images with designated users over a communication network. In some implementations, a method includes causing display of images in a user interface on a first device, each of the images depicting a different person. The images are obtained from a library of images associated with a first user. A selection of a particular image of the images is received based on user input, and a person identifier is determined indicating a particular person depicted in the selected image. The person identifier is designated as a person sharing criterion. A first image not included in the images is obtained and programmatically analyzed to determine that the first image depicts a person that matches the person sharing criterion, and an access permission is updated to grant access to the first image to a second user of a second device over a communication network.