公开(公告)号：US20240265498A1

公开(公告)日：2024-08-08

申请号：US18164458

申请日：2023-02-03

Applicant: Snap inc.

Inventor： Aleksandr Belskikh ,  Georgii Grigorev ,  Pavel Savchenkov

IPC: G06T5/50 ,  G06T3/40 ,  G06T5/00

CPC classification number: G06T5/50 ,  G06T3/4046 ,  G06T5/70 ,  G06T2200/24 ,  G06T2207/20084 ,  G06T2207/20221 ,  G06T2207/30201

Abstract: The subject technology receives an input image and a segmentation mask of the input image. The subject technology obtains reconstructed noise of the input image using the input image and the segmentation mask. The subject technology determines a first set of features by performing a first portion of a forward pass of the reconstructed noise through a decoder. The subject technology determines a second set of features by processing the input image for stable diffusion using an image to image (IMG2IMG) model. The subject technology generates a third set of features based on combining, using the segmentation mask, the first set of features and the second set of features with the reconstructed noise. The subject technology generates an output image by performing a remaining portion of the forward pass of the third set of features through the decoder.