An image editing system accesses an input image displayed via a user interface and generates an instance-aware trimap for the input image by applying an instance-aware image segmentation model to input data including the input image and a segmented image defining a segment of the input image including a first set of pixel values. The trimap defines a modified segment using a second set of pixels different from the first set of pixels. Applying the model includes detecting boundaries of an object depicted in the input image. The second set of pixels is located within the boundaries of the object. Responsive to receiving a request via the user interface, the system generates a modified image by performing an editing operation on the input image including editing a portion of the second set of pixels of the modified segment of the trimap. The system transmits, for display, the modified image.