公开(公告)号：US10789453B2

公开(公告)日：2020-09-29

申请号：US16251436

申请日：2019-01-18

Applicant: SNAP INC.

Inventor： Pavel Savchenkov ,  Dmitry Matov ,  Aleksandr Mashrabov ,  Alexey Pchelnikov

IPC: G06K9/00 ,  G06T11/00 ,  G06Q30/02 ,  G06N3/04

Abstract: Provided are systems and a method for photorealistic real-time face reenactment. An example method includes receiving a target video including a target face and a source video including a source face. The method includes determining, based on the target face, a target facial expression. The method includes determining, based on the source face, a source facial expression. The method includes synthesizing, using the parametric face model, an output face. The output face including the target face wherein the target facial expression is modified to imitate the source facial expression. The method includes generating, based on a deep neural network, mouth and eyes regions, and combining the output face, the mouth, and eyes regions to generate a frame of an output video.

公开(公告)号：US20200234034A1

公开(公告)日：2020-07-23

申请号：US16251436

申请日：2019-01-18

Applicant: SNAP INC.

Inventor： Pavel Savchenkov ,  Dmitry Matov ,  Aleksandr Mashrabov ,  Alexey Pchelnikov

IPC: G06K9/00 ,  G06T11/00 ,  G06N3/04 ,  G06Q30/02

Abstract: Provided are systems and a method for photorealistic real-time face reenactment. An example method includes receiving a target video including a target face and a source video including a source face. The method includes determining, based on the target face, a target facial expression. The method includes determining, based on the source face, a source facial expression. The method includes synthesizing, using the parametric face model, an output face. The output face including the target face wherein the target facial expression is modified to imitate the source facial expression. The method includes generating, based on a deep neural network, mouth and eyes regions, and combining the output face, the mouth, and eyes regions to generate a frame of an output video.

公开(公告)号：US12141933B2

公开(公告)日：2024-11-12

申请号：US18199989

申请日：2023-05-22

Applicant: Snap Inc.

Inventor： Victor Shaburov ,  Alexander Mashrabov ,  Dmitriy Matov ,  Sofia Savinova ,  Alexey Pchelnikov ,  Roman Golobokov

IPC: G06T19/20 ,  G06T13/40 ,  G06V40/16

Abstract: Disclosed are systems and methods for template-based generation of personalized videos. An example method includes receiving a sequence of frame images, face area parameters corresponding to positions of a face area in a frame image of the sequence of frame images, and facial landmark parameters corresponding to the frame image of the sequence of frame images, receiving an image of a source face, modifying, based on the facial landmark parameters corresponding to the frame image, the image of the source face to obtain a further face image featuring the source face adopting a facial expression corresponding to the facial landmark parameters, and inserting the further face image into the frame image at a position determined by the face area parameters corresponding to the frame image, thereby generating an output frame of an output video.

公开(公告)号：US20240071131A1

公开(公告)日：2024-02-29

申请号：US18500803

申请日：2023-11-02

Applicant: Snap Inc.

Inventor： Ivan Babanin ,  Valerii Fisiun ,  Diana Maksimova ,  Alexey Pchelnikov

IPC: G06V40/16 ,  G06N3/045 ,  G06T11/60 ,  G06T19/00

CPC classification number: G06V40/168 ,  G06N3/045 ,  G06T11/60 ,  G06T19/006 ,  G06T2200/24

Abstract: The subject technology displays first augmented reality content on a computing device, the first augmented reality content comprising a first output media content. The subject technology provides for display a plurality of selectable graphical items, each of the selectable graphical items corresponding to a different augmented reality content including a set of media content modified utilizing facial synthesis. The subject technology receives a selection of one of the plurality of selectable graphical items. The subject technology, based at least in part on the selection, identifies second augmented reality content. The subject technology provides the second augmented reality content for display on the computing device.

公开(公告)号：US20220319229A1

公开(公告)日：2022-10-06

申请号：US17704920

申请日：2022-03-25

Applicant: Snap Inc.

Inventor： Ivan Babanin ,  Valerii Fisiun ,  Diana Maksimova ,  Alexey Pchelnikov

IPC: G06V40/16 ,  G06T19/00 ,  G06T11/60 ,  G06N3/04

Abstract: The subject technology displays first augmented reality content on a computing device, the first augmented reality content comprising a first output media content. The subject technology provides for display a plurality of selectable graphical items, each of the selectable graphical items corresponding to a different augmented reality content including a set of media content modified utilizing facial synthesis. The subject technology receives a selection of one of the plurality of selectable graphical items. The subject technology, based at least in part on the selection, identifies second augmented reality content. The subject technology provides the second augmented reality content for display on the computing device.

公开(公告)号：US12148064B2

公开(公告)日：2024-11-19

申请号：US17703716

申请日：2022-03-24

Applicant: Snap Inc.

Inventor： Alexandr Marinenko ,  Aleksandr Mashrabov ,  Alexey Pchelnikov

IPC: G06T11/00 ,  G06Q30/0241 ,  G06Q50/00

Abstract: The subject technology receives frames of a source media content, the frames of the source media content including representations of a head and a face of a source actor. The subject technology generates sets of source pose parameters. The subject technology receives at least one target image, the at least one target image including representations of a target head and a target face of a target entity. The subject technology generates, based at least in part on the sets of source pose parameters, an output media content, each frame of the output media content includes an image of the target face in at least one frame of the output media content. The subject technology provides an online advertisement based at least in part on the output media content for display on a computing device.

公开(公告)号：US11861936B2

公开(公告)日：2024-01-02

申请号：US17869794

申请日：2022-07-21

Applicant: Snap Inc.

Inventor： Pavel Savchenkov ,  Dmitry Matov ,  Aleksandr Mashrabov ,  Alexey Pchelnikov

IPC: G06V40/16 ,  G06N3/04 ,  G06Q30/0251 ,  G06T11/00

CPC classification number: G06V40/161 ,  G06N3/04 ,  G06Q30/0254 ,  G06Q30/0269 ,  G06T11/001 ,  G06V40/174 ,  G06V40/178

Abstract: Provided are systems and methods for face reenactment. An example method includes receiving visual data including a visible portion of a source face, determining, based on the visible portion of the source face, a first portion of source face parameters associated with a parametric face model, where the first portion corresponds to the visible portion, predicting, based partially on the visible portion of the source face, a second portion of the source face parameters, where the second portion corresponds to the rest of the source face, receiving a target video that includes a target face, determining, based on the target video, target face parameters associated with the parametric face model and corresponding to the target face, and synthesizing, using the parametric face model, based on the source face parameters and the target face parameters, an output face that includes the source face imitating a facial expression of the target face.

公开(公告)号：US20220358784A1

公开(公告)日：2022-11-10

申请号：US17869794

申请日：2022-07-21

Applicant: Snap Inc.

Inventor： Pavel Savchenkov ,  Dmitry Matov ,  Aleksandr Mashrabov ,  Alexey Pchelnikov

IPC: G06V40/16 ,  G06N3/04 ,  G06Q30/02 ,  G06T11/00

Abstract: Provided are systems and methods for face reenactment. An example method includes receiving visual data including a visible portion of a source face, determining, based on the visible portion of the source face, a first portion of source face parameters associated with a parametric face model, where the first portion corresponds to the visible portion, predicting, based partially on the visible portion of the source face, a second portion of the source face parameters, where the second portion corresponds to the rest of the source face, receiving a target video that includes a target face, determining, based on the target video, target face parameters associated with the parametric face model and corresponding to the target face, and synthesizing, using the parametric face model, based on the source face parameters and the target face parameters, an output face that includes the source face imitating a facial expression of the target face.

公开(公告)号：US20200234508A1

公开(公告)日：2020-07-23

申请号：US16661086

申请日：2019-10-23

Applicant: SNAP INC.

Inventor： Victor Shaburov ,  Alexander Mashrabov ,  Dmitriy Matov ,  Sofia Savinova ,  Alexey Pchelnikov ,  Roman Golobkov

IPC: G06T19/20 ,  G06K9/00 ,  G06T13/40

Abstract: Disclosed are systems and methods for template-based generation of personalized videos. An example method may commence with receiving video configuration data including a sequence of frame images, a sequence of face area parameters defining positions of a face area in the frame images, and a sequence of facial landmark parameters defining positions of facial landmarks in the frame images. The method may continue with receiving an image of a source face. The method may further include generating an output video. The generation of the output video may include modifying a frame image of the sequence of frame images. Specifically, the image of the source face may be modified to obtain a further image featuring the source face adopting a facial expression corresponding to the facial landmark parameters. The further image may be inserted into the frame image at a position determined by face area parameters corresponding to the frame image.

公开(公告)号：US20250045979A1

公开(公告)日：2025-02-06

申请号：US18923285

申请日：2024-10-22

Applicant: Snap Inc.

Inventor： Alexandr Marinenko ,  Aleksandr Mashrabov ,  Alexey Pchelnikov

IPC: G06T11/00 ,  G06Q30/0241 ,  G06Q50/00

Abstract: The subject technology receives frames of a source media content, the frames of the source media content including representations of a head and a face of a source actor. The subject technology generates sets of source pose parameters. The subject technology receives at least one target image, the at least one target image including representations of a target head and a target face of a target entity. The subject technology generates, based at least in part on the sets of source pose parameters, an output media content, each frame of the output media content includes an image of the target face in at least one frame of the output media content. The subject technology provides an online advertisement based at least in part on the output media content for display on a computing device.