公开(公告)号：US11410457B2

公开(公告)日：2022-08-09

申请号：US17034029

申请日：2020-09-28

Applicant: Snap Inc.

Inventor： Pavel Savchenkov ,  Dmitry Matov ,  Aleksandr Mashrabov ,  Alexey Pchelnikov

IPC: G06V40/16 ,  G06N3/04 ,  G06Q30/02 ,  G06T11/00

Abstract: Provided are systems and a method for photorealistic real-time face reenactment. An example method includes receiving a target video including a target face and a scenario including a series of source facial expressions, determining, based on the target face, one or more target facial expressions, and synthesizing, using the parametric face model, an output face. The output face includes the target face. The one or more target facial expressions are modified to imitate the source facial expressions. The method further includes generating, based on a deep neural network, a mouth region and an eyes region, and combining the output face, the mouth region, and the eyes region to generate a frame of an output video.

公开(公告)号：US20220147566A1

公开(公告)日：2022-05-12

申请号：US17581350

申请日：2022-01-21

Applicant: Snap Inc.

Inventor： Jeremy Voss ,  Victor Shaburov ,  Aleksandr Mashrabov ,  Dmitriy Matov ,  Hanna Rulevska ,  Dmytro Ishchenko

IPC: G06F16/732 ,  G06F16/71

Abstract: A system for searching and ranking modifiable videos is provided. In one example embodiment, the system includes a at least one processor and a memory storing processor-executable codes, wherein the at least one processor is configured to implement the following operations upon executing the processor-executable codes: processing a first set of search requests by a first search rule to obtain a first set of first results; processing a second set of search requests by a second search rule to obtain a second set of second results; determining a first metric based, at least partially, on the first set of first results; determining a second metric based at least partially on the second set of second results; selecting, based on the first metric and the second metric, a search rule from the first search rule and the second search rule; and configuring a search engine to apply the search rule.

公开(公告)号：US11263260B2

公开(公告)日：2022-03-01

申请号：US16835656

申请日：2020-03-31

Applicant: Snap Inc.

Inventor： Jeremy Voss ,  Victor Shaburov ,  Aleksandr Mashrabov ,  Dmitriy Matov ,  Hanna Rulevska ,  Dmytro Ishchenko

IPC: G06F16/732 ,  G06F16/71 ,  H04L12/58 ,  H04L51/10

Abstract: A system for searching and ranking modifiable videos in a multimedia messaging application (MMA) is provided. In one example embodiment, the system includes a database configured to store modifiable videos, the modifiable videos being associated with text messages and rankings, a processor, and a memory storing processor-executable codes, wherein the processor is configured to implement the following operations upon executing the processor-executable codes: receiving, via the MMA, an input of a user; selecting, based on the input, a list of relevant modifiable videos from the database; rendering, via the MMA, the list of relevant modifiable videos for viewing by the user; determining that the user has shared, via the MMA, a modifiable video from the list; storing an information concerning the list and the shared modifiable video into a statistical log; and updating, based on the information in the statistical log, the rankings of the modifiable videos in the database.

公开(公告)号：US11212383B2

公开(公告)日：2021-12-28

申请号：US16878868

申请日：2020-05-20

Applicant: Snap Inc.

Inventor： Jeremy Voss ,  Victor Shaburov ,  Ivan Babanin ,  Aleksandr Mashrabov ,  Roman Golobokov

IPC: H04B1/38 ,  H04M1/72439 ,  H04L12/58 ,  G06T11/60 ,  G06T13/80 ,  H04M1/72427 ,  H04M1/72436 ,  H04M1/72442

Abstract: A system for customizing text messages in modifiable videos of a multimedia messaging application (MMA) is provided. In one example embodiment, the system includes a processor and a memory storing processor-executable codes, wherein the processor is configured to analyze recent messages of a user in the MMA to determine a context of the recent messages; determine, based on the context, a customized text message; select, based on the context, a list of relevant modifiable videos from a database configured to store modifiable videos, the modifiable videos being associated with preset text messages; replace the preset text messages in the relevant modifiable videos with the customized message; and render the list of relevant modifiable videos for viewing and selecting by the user, the rendering including displaying the customized text message in the relevant modifiable videos.

公开(公告)号：US11049310B2

公开(公告)日：2021-06-29

申请号：US16251472

申请日：2019-01-18

Applicant: SNAP INC.

Inventor： Eugene Krokhalev ,  Aleksandr Mashrabov ,  Pavel Savchenkov

IPC: G06T13/80 ,  G06T7/174 ,  G06K9/00

Abstract: Provided are systems and methods for photorealistic real-time portrait animation. An example method includes receiving a scenario video with at least one input frame. The input frame includes a first face. The method further includes receiving a target image with a second face. The method further includes determining, based on the at least one input frame and the target image, two-dimensional (2D) deformations, wherein the 2D deformations, when applied to the second face, modify the second face to imitate at least a facial expression and a head orientation of the first face. The method further includes applying, by the computing device, the 2D deformations to the target image to obtain at least one output frame of an output video.

公开(公告)号：US20200234482A1

公开(公告)日：2020-07-23

申请号：US16251472

申请日：2019-01-18

Applicant: SNAP INC.

Inventor： Eugene Krokhalev ,  Aleksandr Mashrabov ,  Pavel Savchenkov

IPC: G06T13/80 ,  G06T7/174 ,  G06K9/00

Abstract: Provided are systems and methods for photorealistic real-time portrait animation. An example method includes receiving a scenario video with at least one input frame. The input frame includes a first face. The method further includes receiving a target image with a second face. The method further includes determining, based on the at least one input frame and the target image, two-dimensional (2D) deformations, wherein the 2D deformations, when applied to the second face, modify the second face to imitate at least a facial expression and a head orientation of the first face. The method further includes applying, by the computing device, the 2D deformations to the target image to obtain at least one output frame of an output video.

公开(公告)号：US12277640B2

公开(公告)日：2025-04-15

申请号：US18641472

申请日：2024-04-22

Applicant: Snap Inc.

Inventor： Eugene Krokhalev ,  Aleksandr Mashrabov ,  Pavel Savchenkov

IPC: G06T13/80 ,  G06T7/174 ,  G06V40/16

Abstract: Provided are systems and methods for portrait animation. An example method includes receiving, by a computing device, scenario data including information concerning movements of a first head, receiving, by the computing device, a target image including a second head and a background, determining, by the computing device and based on the target image and the information concerning the movements of the first head, two-dimensional (2D) deformations of the second head in the target image, applying, by the computing device, the 2D deformations to the target image to obtain at least one output frame of an output video, the at least one output frame including the second head displaced according to the movements of the first head, and filling, by the computing device and using a background prediction neural network, a portion of the background in gaps between the displaced second head and the background.

公开(公告)号：US20250119492A1

公开(公告)日：2025-04-10

申请号：US18989143

申请日：2024-12-20

Applicant: Snap Inc.

Inventor： Jeremy Voss ,  Victor Shaburov ,  Ivan Babanin ,  Aleksandr Mashrabov ,  Roman Golobokov

IPC: H04M1/72439 ,  G06T11/60 ,  G06T13/80 ,  H04L51/02 ,  H04L51/216 ,  H04M1/72427 ,  H04M1/72436 ,  H04M1/72442

Abstract: Provided are systems and methods for customizing modifiable videos. An example method includes analyzing recent messages associated with a user in a multimedia messaging application to determine an emotional state of the user, determining, based on the emotional state, a customized property of a modifiable feature of a modifiable video, where the modifiable video includes a preset property of the modifiable feature, replacing the preset property of the modifiable feature with the customized property of the modifiable feature, providing a user interface enabling the user to view the modifiable video and modify the customized property of the modifiable feature to a modified property, and, upon determining that the user has modified the customized property, storing information concerning the emotional state, the customized property, and the modified property to a statistical log.

公开(公告)号：US12243348B2

公开(公告)日：2025-03-04

申请号：US18534258

申请日：2023-12-08

Applicant: Snap Inc.

Inventor： Roman Golobokov ,  Alexandr Marinenko ,  Aleksandr Mashrabov ,  Aleksei Bromot ,  Grigoriy Tkachenko

IPC: G06V40/16 ,  G06T13/40 ,  G06T17/00 ,  G06T19/00 ,  G06T19/20

Abstract: The subject technology captures first image data by a computing device, the first image data comprising a target face of a target actor and facial expressions of the target actor, the facial expressions including lip movements. The subject technology generates, based at least in part on frames of a source media content, sets of source pose parameters. The subject technology receives a selection of a particular facial expression from a set of facial expressions. The subject technology generates, based at least in part on sets of source pose parameters and the selection of the particular facial expression, an output media content. The subject technology provides augmented reality content based at least in part on the output media content for display on the computing device.

公开(公告)号：US20240312099A1

公开(公告)日：2024-09-19

申请号：US18673574

申请日：2024-05-24

Applicant: Snap Inc.

Inventor： Sergei Gorbatyuk ,  Nikolai Smirnov ,  Aleksandr Mashrabov ,  Egor Nemchinov

IPC: G06T13/40 ,  G06F3/0482 ,  G06F3/04845 ,  G06T7/194 ,  G06T7/20 ,  G06T7/70 ,  G06T19/00

CPC classification number: G06T13/40 ,  G06F3/0482 ,  G06F3/04845 ,  G06T7/194 ,  G06T7/20 ,  G06T7/70 ,  G06T19/00 ,  G06T2200/24 ,  G06T2207/10016 ,  G06T2207/30196

Abstract: Provided are systems and methods for animating a single image of a human body and applying effects. An example method includes receiving an image of a body, receiving, via a user interface, a user input including parameters associated with a motion, fitting a portion of the image of the body to a hair model designed to generate a hair image corresponding to the motion, where the portion of the image of the body includes hair, and generating a video featuring the body repeating the motion, where the generation of the video is based on the image of the body, the parameters associated with the motion, and the hair image generated by the hair model.