公开(公告)号：US12277639B2

公开(公告)日：2025-04-15

申请号：US18149007

申请日：2022-12-30

Applicant: Snap Inc.

Inventor： Aleksandr Belskikh ,  Menglei Chai ,  Antoine Chassang ,  Anna Kovalenko ,  Pavel Savchenkov

IPC: G06T13/40 ,  G06N3/094 ,  G06T19/00

Abstract: Embodiments enable virtual hair generation. The virtual hair generation can be performed by generating a first image of a face using a GAN model, applying 3D virtual hair on the first image to generate a second image with 3D virtual hair, projecting the second image with 3D virtual hair into a GAN latent space to generate a third image with virtual hair, performing a blend of the virtual hair with the first image of the face to generate a new image with new virtual hair that corresponds to the 3D virtual hair, training a neural network that receives the second image with the 3D virtual hair and provides an output image with virtual hair, and generating using the trained neural network, a particular output image with hair based on a particular input image with 3D virtual hair.

公开(公告)号：US20240265498A1

公开(公告)日：2024-08-08

申请号：US18164458

申请日：2023-02-03

Applicant: Snap inc.

Inventor： Aleksandr Belskikh ,  Georgii Grigorev ,  Pavel Savchenkov

IPC: G06T5/50 ,  G06T3/40 ,  G06T5/00

CPC classification number: G06T5/50 ,  G06T3/4046 ,  G06T5/70 ,  G06T2200/24 ,  G06T2207/20084 ,  G06T2207/20221 ,  G06T2207/30201

Abstract: The subject technology receives an input image and a segmentation mask of the input image. The subject technology obtains reconstructed noise of the input image using the input image and the segmentation mask. The subject technology determines a first set of features by performing a first portion of a forward pass of the reconstructed noise through a decoder. The subject technology determines a second set of features by processing the input image for stable diffusion using an image to image (IMG2IMG) model. The subject technology generates a third set of features based on combining, using the segmentation mask, the first set of features and the second set of features with the reconstructed noise. The subject technology generates an output image by performing a remaining portion of the forward pass of the third set of features through the decoder.

公开(公告)号：US20240221259A1

公开(公告)日：2024-07-04

申请号：US18149007

申请日：2022-12-30

Applicant: Snap Inc.

Inventor： Aleksandr Belskikh ,  Menglei Chai ,  Antoine Chassang ,  Anna Kovalenko ,  Pavel Savchenkov

IPC: G06T13/40 ,  G06N3/094 ,  G06T19/00

CPC classification number: G06T13/40 ,  G06N3/094 ,  G06T19/006

Abstract: The subject technology generates a first image of a face using a GAN model. The subject technology applies 3D virtual hair on the first image to generate a second image with 3D virtual hair. The subject technology projects the second image with 3D virtual hair into a GAN latent space to generate a third image with realistic virtual hair. The subject technology performs a blend of the realistic virtual hair with the first image of the face to generate a new image with new realistic hair that corresponds to the 3D virtual hair. The subject technology trains a neural network that receives the second image with the 3D virtual hair and provides an output image with realistic virtual hair. The subject technology generates using the trained neural network, a particular output image with realistic hair based on a particular input image with 3D virtual hair.

公开(公告)号：US20250022264A1

公开(公告)日：2025-01-16

申请号：US18221241

申请日：2023-07-12

Applicant: Snap Inc.

Inventor： Aleksandr Belskikh ,  Antoine Chassang ,  Anna Kovalenko ,  Pavel Savchenkov

IPC: G06V10/82 ,  G06T3/00 ,  G06V20/40

Abstract: The subject technology trains a neural network based on a training process. The subject technology selects a frame from an input video, the selected frame comprising image data including a representation of a face and hair, the representation of the hair being masked. The subject technology determines a previous predicted frame. The subject technology concatenates the selected frame and the previous predicted frame to generate a concatenated frame, the concatenated frame being provided to the neural network. The subject technology generates, using the neural network, a set of outputs including an output tensor, warping field, and a soft mask. The subject technology performs, using a warping field, a warp of the selected frame and the output tensor. The subject technology generates a prediction corresponding to a corrected texture rendering of the selected frame.