公开(公告)号：US20250022246A1

公开(公告)日：2025-01-16

申请号：US18904104

申请日：2024-10-02

Applicant: Snap Inc.

Inventor： Victor Shaburov ,  Alexander Mashrabov ,  Dmitriy Matov ,  Sofia Savinova ,  Alexey Pchelnikov ,  Roman Golobokov

IPC: G06T19/20 ,  G06T13/40 ,  G06V40/16

Abstract: Provided are systems and methods for template-based generation of personalized videos. An example method includes receiving a sequence of frame images, face area parameters corresponding to positions of a face area in a frame image of the sequence of frame images, and facial landmark parameters corresponding to the frame image of the sequence of frame images, where the facial landmark parameters are absent from the frame images, receiving an image of a source face, modifying, based on the facial landmark parameters corresponding to the frame image, the image of the source face to obtain a further face image featuring the source face adopting a facial expression corresponding to the facial landmark parameters, and inserting the further face image into the frame image at a position determined by the face area parameters corresponding to the frame image, thereby generating an output frame of an output video.

公开(公告)号：US11694417B2

公开(公告)日：2023-07-04

申请号：US17674876

申请日：2022-02-18

Applicant: Snap Inc.

Inventor： Victor Shaburov ,  Alexander Mashrabov ,  Dmitriy Matov ,  Sofia Savinova ,  Alexey Pchelnikov ,  Roman Golobokov

IPC: G06T19/20 ,  G06T13/40 ,  G06V40/16

CPC classification number: G06T19/20 ,  G06T13/40 ,  G06V40/161 ,  G06V40/171 ,  G06V40/172 ,  G06V40/174

Abstract: Disclosed are systems and methods for template-based generation of personalized videos. An example method may commence with receiving video configuration data including a sequence of frame images, a sequence of face area parameters defining positions of a face area in the frame images, and a sequence of skin masks defining positions of a skin area of a part of the at least one body in the frame images. The method may continue with receiving an image of a source face. The method may further include determining color data associated with the source face. The method may include recoloring the skin area of the part of the at least one body in the frame image and inserting the image of the source face into the frame image at a position determined by face area parameters corresponding to the frame image to generate an output frame of an output video.

公开(公告)号：US20220319060A1

公开(公告)日：2022-10-06

申请号：US17703716

申请日：2022-03-24

Applicant: Snap Inc

Inventor： Alexandr Marinenko ,  Aleksandr Mashrabov ,  Alexey Pchelnikov

IPC: G06T11/00 ,  G06Q30/02

Abstract: The subject technology receives frames of a source media content, the frames of the source media content including representations of a head and a face of a source actor. The subject technology generates sets of source pose parameters. The subject technology receives at least one target image, the at least one target image including representations of a target head and a target face of a target entity. The subject technology generates, based at least in part on the sets of source pose parameters, an output media content, each frame of the output media content includes an image of the target face in at least one frame of the output media content. The subject technology provides an online advertisement based at least in part on the output media content for display on a computing device.

公开(公告)号：US20210012090A1

公开(公告)日：2021-01-14

申请号：US17034029

申请日：2020-09-28

Applicant: Snap Inc.

Inventor： Pavel Savchenkov ,  Dmitry Matov ,  Aleksandr Mashrabov ,  Alexey Pchelnikov

IPC: G06K9/00 ,  G06N3/04 ,  G06Q30/02 ,  G06T11/00

Abstract: Provided are systems and a method for photorealistic real-time face reenactment. An example method includes receiving a target video including a target face and a scenario including a series of source facial expressions, determining, based on the target face, one or more target facial expressions, and synthesizing, using the parametric face model, an output face. The output face includes the target face. The one or more target facial expressions are modified to imitate the source facial expressions. The method further includes generating, based on a deep neural network, a mouth region and an eyes region, and combining the output face, the mouth region, and the eyes region to generate a frame of an output video.

公开(公告)号：US12148244B2

公开(公告)日：2024-11-19

申请号：US18500803

申请日：2023-11-02

Applicant: Snap Inc.

Inventor： Ivan Babanin ,  Valerii Fisiun ,  Diana Maksimova ,  Alexey Pchelnikov

IPC: G06T11/60 ,  G06N3/045 ,  G06T19/00 ,  G06V40/16

Abstract: The subject technology displays first augmented reality content on a computing device, the first augmented reality content comprising a first output media content. The subject technology provides for display a plurality of selectable graphical items, each of the selectable graphical items corresponding to a different augmented reality content including a set of media content modified utilizing facial synthesis. The subject technology receives a selection of one of the plurality of selectable graphical items. The subject technology, based at least in part on the selection, identifies second augmented reality content. The subject technology provides the second augmented reality content for display on the computing device.

公开(公告)号：US20240078838A1

公开(公告)日：2024-03-07

申请号：US18509502

申请日：2023-11-15

Applicant: Snap Inc.

Inventor： Pavel Savchenkov ,  Dmitry Matov ,  Aleksandr Mashrabov ,  Alexey Pchelnikov

IPC: G06V40/16 ,  G06N3/04 ,  G06Q30/0251 ,  G06T11/00

CPC classification number: G06V40/161 ,  G06N3/04 ,  G06Q30/0254 ,  G06Q30/0269 ,  G06T11/001 ,  G06V40/174 ,  G06V40/178

Abstract: Provided are systems and methods for face reenactment. An example method includes receiving a target video that includes at least one target frame, where the at least one target frame includes a target face, receiving a scenario including a series of source facial expressions, determining, based on the target face, a target facial expression of the target face, synthesizing, based on a parametric face model and a texture model, an output face including the target face, where the target facial expression of the target face is modified to imitate a source facial expression of the series of source facial expressions, and generating, based on the output face, a frame of an output video. The parametric face model includes a template mesh pre-generated based on historical images of faces of a plurality of individuals, where the template mesh includes a pre-determined number of vertices.

公开(公告)号：US11816926B2

公开(公告)日：2023-11-14

申请号：US17704920

申请日：2022-03-25

Applicant: Snap Inc.

Inventor： Ivan Babanin ,  Valerii Fisiun ,  Diana Maksimova ,  Alexey Pchelnikov

IPC: G06V40/16 ,  G06T11/60 ,  G06T19/00 ,  G06N3/045

CPC classification number: G06V40/168 ,  G06N3/045 ,  G06T11/60 ,  G06T19/006 ,  G06T2200/24

Abstract: The subject technology displays first augmented reality content on a computing device, the first augmented reality content comprising a first output media content. The subject technology provides for display a plurality of selectable graphical items, each of the selectable graphical items corresponding to a different augmented reality content including a set of media content modified utilizing facial synthesis. The subject technology receives a selection of one of the plurality of selectable graphical items. The subject technology, based at least in part on the selection, identifies second augmented reality content. The subject technology provides the second augmented reality content for display on the computing device.

公开(公告)号：US20230290098A1

公开(公告)日：2023-09-14

申请号：US18199989

申请日：2023-05-22

Applicant: Snap Inc.

Inventor： Victor Shaburov ,  Alexander Mashrabov ,  Dmitriy Matov ,  Sofia Savinova ,  Alexey Pchelnikov ,  Roman Golobokov

IPC: G06T19/20 ,  G06T13/40 ,  G06V40/16

CPC classification number: G06T19/20 ,  G06T13/40 ,  G06V40/161 ,  G06V40/171 ,  G06V40/172 ,  G06V40/174

Abstract: Disclosed are systems and methods for template-based generation of personalized videos. An example method includes receiving a sequence of frame images, face area parameters corresponding to positions of a face area in a frame image of the sequence of frame images, and facial landmark parameters corresponding to the frame image of the sequence of frame images, receiving an image of a source face, modifying, based on the facial landmark parameters corresponding to the frame image, the image of the source face to obtain a further face image featuring the source face adopting a facial expression corresponding to the facial landmark parameters, and inserting the further face image into the frame image at a position determined by the face area parameters corresponding to the frame image, thereby generating an output frame of an output video.

公开(公告)号：US11410457B2

公开(公告)日：2022-08-09

申请号：US17034029

申请日：2020-09-28

Applicant: Snap Inc.

Inventor： Pavel Savchenkov ,  Dmitry Matov ,  Aleksandr Mashrabov ,  Alexey Pchelnikov

IPC: G06V40/16 ,  G06N3/04 ,  G06Q30/02 ,  G06T11/00

Abstract: Provided are systems and a method for photorealistic real-time face reenactment. An example method includes receiving a target video including a target face and a scenario including a series of source facial expressions, determining, based on the target face, one or more target facial expressions, and synthesizing, using the parametric face model, an output face. The output face includes the target face. The one or more target facial expressions are modified to imitate the source facial expressions. The method further includes generating, based on a deep neural network, a mouth region and an eyes region, and combining the output face, the mouth region, and the eyes region to generate a frame of an output video.

公开(公告)号：US20220172449A1

公开(公告)日：2022-06-02

申请号：US17674876

申请日：2022-02-18

Applicant: Snap Inc.

Inventor： Victor Shaburov ,  Alexander Mashrabov ,  Dmitriy Matov ,  Sofia Savinova ,  Alexey Pchelnikov ,  Roman Golobokov

IPC: G06T19/20 ,  G06T13/40 ,  G06V40/16

Abstract: Disclosed are systems and methods for template-based generation of personalized videos. An example method may commence with receiving video configuration data including a sequence of frame images, a sequence of face area parameters defining positions of a face area in the frame images, and a sequence of skin masks defining positions of a skin area of a part of the at least one body in the frame images. The method may continue with receiving an image of a source face. The method may further include determining color data associated with the source face. The method may include recoloring the skin area of the part of the at least one body in the frame image and inserting the image of the source face into the frame image at a position determined by face area parameters corresponding to the frame image to generate an output frame of an output video.