公开(公告)号：US20240371085A1

公开(公告)日：2024-11-07

申请号：US18770353

申请日：2024-07-11

Applicant: Snap Inc.

Inventor： Gleb Dmukhin ,  Egor Nemchinov ,  Yurii Volkov

IPC: G06T15/50 ,  G06N3/08 ,  G06V10/60

Abstract: A messaging system performs image processing to estimate lighting properties with neural networks for images provided by users of the messaging system. A method of estimating light properties includes receiving an input image with first lighting properties and processing the input image using a convolutional neural network to generate an estimate of the first lighting properties. The method may further include modifying the input image with an augmentation to generate a modified input image, where the augmentation has second lighting properties, and changing the second lighting properties of the augmentation in the modified input image to the estimate of the first lighting properties.

公开(公告)号：US12125147B2

公开(公告)日：2024-10-22

申请号：US17663176

申请日：2022-05-12

Applicant: Snap Inc.

Inventor： Pavel Savchenkov ,  Yurii Volkov ,  Jeremy Baker Voss

IPC: G06T19/00 ,  G06N20/00 ,  G06T13/40 ,  H04L51/046

CPC classification number: G06T19/006 ,  G06N20/00 ,  G06T13/40 ,  H04L51/046 ,  G06T2200/24

Abstract: A methodology for training a machine learning model to generate color-neutral input face images is described. For each training face image from a training dataset that is used for training the model, the training system generates an input face image, which has the color and lighting of a randomly selected image from the set of color source images, and which has facial features and expression of a face object from the training face image. Because, during training, the machine learning model is “confused” by changing the color and lighting of a training face image to a randomly selected different color and lighting, the trained machine learning model generates a color neutral embedding representing facial features from the training face image.

公开(公告)号：US20220392133A1

公开(公告)日：2022-12-08

申请号：US17881947

申请日：2022-08-05

Applicant: Snap Inc.

Inventor： Yurii Volkov ,  Pavel Savchenkov ,  Nikolai Smirnov ,  Aleksandr Mashrabov

IPC: G06T13/40 ,  G06T7/70 ,  G06V40/16

Abstract: Provided are systems and methods for realistic head turns and face animation synthesis. An example method includes receiving a source frame of a source video, where the source frame includes a head and a face of a source actor, generating source pose parameters corresponding to a pose of the head and a facial expression of the source actor; receiving a target image including a target head and a target face of a target person, determining target identity information associated with the target head and the target face of the target person, replacing source identity information in the source pose parameters with the target identity information to obtain further source pose parameters, and generating an output frame of an output video that includes a modified image of the target face and the target head adopting the pose of the head and the facial expression of the source actor.

公开(公告)号：US20220270332A1

公开(公告)日：2022-08-25

申请号：US17663176

申请日：2022-05-12

Applicant: Snap Inc.

Inventor： Pavel Savchenkov ,  Yurii Volkov ,  Jeremy Baker Voss

IPC: G06T19/00 ,  G06N20/00 ,  G06T13/40 ,  H04L51/046

Abstract: A methodology for training a machine learning model to generate color-neutral input face images is described. For each training face image from a training dataset that is used for training the model, the training system generates an input face image, which has the color and lighting of a randomly selected image from the set of color source images, and which has facial features and expression of a face object from the training face image. Because, during training, the machine learning model is “confused” by changing the color and lighting of a training face image to a randomly selected different color and lighting, the trained machine learning model generates a color neutral embedding representing facial features from the training face image.

公开(公告)号：US11335069B1

公开(公告)日：2022-05-17

申请号：US17107410

申请日：2020-11-30

Applicant: Snap Inc.

Inventor： Pavel Savchenkov ,  Yurii Volkov ,  Jeremy Baker Voss

IPC: G06T19/00 ,  G06T13/40 ,  G06N20/00 ,  H04L51/046

Abstract: In some embodiments, users' experience of engaging with augmented reality technology is enhanced by providing a process, referred to as face animation synthesis, that replaces an actor's face in the frames of a video with a user's face from the user's portrait image. The resulting face in the frames of the video retains the facial expressions, as well as color and lighting, of the actor's face but, at the same time, has the likeness of the user's face. An example face animation synthesis experience can be made available to uses of a messaging system by providing a face animation synthesis augmented reality component.

公开(公告)号：US20200234480A1

公开(公告)日：2020-07-23

申请号：US16662743

申请日：2019-10-24

Applicant: SNAP INC.

Inventor： Yurii Volkov ,  Pavel Savchenkov ,  Maxim Lukin ,  Ivan Belonogov ,  Nikolai Smirnov ,  Aleksandr Mashrabov

IPC: G06T13/40 ,  G06K9/00 ,  G06T7/70

Abstract: Provided are systems and methods for realistic head turns and face animation synthesis. An example method may include receiving frames of a source video with the head and the face of a source actor. The method may then proceed with generating sets of source pose parameters that represent positions of the head and facial expressions of the source actor. The method may further include receiving at least one target image including the target head and the target face of a target person, determining target identity information associated with the target face, and generating an output video based on the target identity information and the sets of source pose parameters. Each frame of the output video can include an image of the target face modified to mimic at least one of the positions of the head of the source actor and at least one of facial expressions of the source actor.