公开(公告)号：US20240414108A1

公开(公告)日：2024-12-12

申请号：US18677723

申请日：2024-05-29

Applicant: Snap Inc.

Inventor： Haowen Sun ,  William Spencer Mulligan ,  Nathan Kenneth Boyd ,  Hee Hun Kim ,  Dmytro Ishchenko ,  Lily Hinkeldey ,  Jason Brewer ,  Charles Melbye ,  Aleksandr Mashrabov

IPC: H04L51/02 ,  G06F16/34 ,  H04L51/216

Abstract: A chatbot system for an interactive platform is disclosed. The chatbot system retrieves a conversation history of one or more conversations between a user and a chatbot from a conversation history datastore and generates one or more summarized memories using the conversation history. One or more moderated memories are generated using the summarized memories. The moderated memories are stored in a memories datastore. A user prompt is received, and a current conversation context is generated from a current conversation between the user and the chatbot. One or more memories are retrieved from the memories datastore using the current conversation context. An augmented prompt is generated using the user prompt and the one or more memories, which is communicated to a generative AI model. A response is received from the generative AI model to the augmented prompt, which is provided to the user.

公开(公告)号：US12148064B2

公开(公告)日：2024-11-19

申请号：US17703716

申请日：2022-03-24

Applicant: Snap Inc.

Inventor： Alexandr Marinenko ,  Aleksandr Mashrabov ,  Alexey Pchelnikov

IPC: G06T11/00 ,  G06Q30/0241 ,  G06Q50/00

Abstract: The subject technology receives frames of a source media content, the frames of the source media content including representations of a head and a face of a source actor. The subject technology generates sets of source pose parameters. The subject technology receives at least one target image, the at least one target image including representations of a target head and a target face of a target entity. The subject technology generates, based at least in part on the sets of source pose parameters, an output media content, each frame of the output media content includes an image of the target face in at least one frame of the output media content. The subject technology provides an online advertisement based at least in part on the output media content for display on a computing device.

公开(公告)号：US20240296614A1

公开(公告)日：2024-09-05

申请号：US18641472

申请日：2024-04-22

Applicant: Snap Inc.

Inventor： Eugene Krokhalev ,  Aleksandr Mashrabov ,  Pavel Savchenkov

IPC: G06T13/80 ,  G06T7/174 ,  G06V40/16

CPC classification number: G06T13/80 ,  G06T7/174 ,  G06V40/167 ,  G06T2207/20084

Abstract: Provided are systems and methods for portrait animation. An example method includes receiving, by a computing device, scenario data including information concerning movements of a first head, receiving, by the computing device, a target image including a second head and a background, determining, by the computing device and based on the target image and the information concerning the movements of the first head, two-dimensional (2D) deformations of the second head in the target image, applying, by the computing device, the 2D deformations to the target image to obtain at least one output frame of an output video, the at least one output frame including the second head displaced according to the movements of the first head, and filling, by the computing device and using a background prediction neural network, a portion of the background in gaps between the displaced second head and the background.

公开(公告)号：US11875600B2

公开(公告)日：2024-01-16

申请号：US17706830

申请日：2022-03-29

Applicant: Snap Inc.

Inventor： Roman Golobokov ,  Alexandr Marinenko ,  Aleksandr Mashrabov ,  Aleksei Bromot ,  Grigoriy Tkachenko

IPC: G06V40/16 ,  G06T19/00 ,  G06T13/40 ,  G06T17/00

CPC classification number: G06V40/168 ,  G06T19/006 ,  G06V40/174 ,  G06T2200/24

Abstract: The subject technology captures first image data by a computing device, the first image data comprising a target face of a target actor and facial expressions of the target actor, the facial expressions including lip movements. The subject technology generates, based at least in part on frames of a source media content, sets of source pose parameters. The subject technology receives a selection of a particular facial expression from a set of facial expressions. The subject technology generates, based at least in part on sets of source pose parameters and the selection of the particular facial expression, an output media content. The subject technology provides augmented reality content based at least in part on the output media content for display on the computing device.

公开(公告)号：US11861936B2

公开(公告)日：2024-01-02

申请号：US17869794

申请日：2022-07-21

Applicant: Snap Inc.

Inventor： Pavel Savchenkov ,  Dmitry Matov ,  Aleksandr Mashrabov ,  Alexey Pchelnikov

IPC: G06V40/16 ,  G06N3/04 ,  G06Q30/0251 ,  G06T11/00

CPC classification number: G06V40/161 ,  G06N3/04 ,  G06Q30/0254 ,  G06Q30/0269 ,  G06T11/001 ,  G06V40/174 ,  G06V40/178

Abstract: Provided are systems and methods for face reenactment. An example method includes receiving visual data including a visible portion of a source face, determining, based on the visible portion of the source face, a first portion of source face parameters associated with a parametric face model, where the first portion corresponds to the visible portion, predicting, based partially on the visible portion of the source face, a second portion of the source face parameters, where the second portion corresponds to the rest of the source face, receiving a target video that includes a target face, determining, based on the target video, target face parameters associated with the parametric face model and corresponding to the target face, and synthesizing, using the parametric face model, based on the source face parameters and the target face parameters, an output face that includes the source face imitating a facial expression of the target face.

公开(公告)号：US11568589B2

公开(公告)日：2023-01-31

申请号：US17751796

申请日：2022-05-24

Applicant: Snap Inc.

Inventor： Eugene Krokhalev ,  Aleksandr Mashrabov ,  Pavel Savchenkov

IPC: G06T13/80 ,  G06T7/174 ,  G06V40/16

Abstract: Disclosed are systems and methods for portrait animation. An example method includes receiving, by a computing device, a scenario video, where the scenario video includes at least one input frame and the at least one input frame includes a first face, receiving, by the computing device, a target image, where the target image includes a second face, determining, by the computing device and based on the at least one input frame and the target image, two-dimensional (2D) deformations of the second face in the target image, where the 2D deformations, when applied to the second face, modify the second face to imitate at least a facial expression of the first face, and applying, by the computing device, the 2D deformations to the target image to obtain at least one output frame of an output video.

公开(公告)号：US20220358784A1

公开(公告)日：2022-11-10

申请号：US17869794

申请日：2022-07-21

Applicant: Snap Inc.

Inventor： Pavel Savchenkov ,  Dmitry Matov ,  Aleksandr Mashrabov ,  Alexey Pchelnikov

IPC: G06V40/16 ,  G06N3/04 ,  G06Q30/02 ,  G06T11/00

Abstract: Provided are systems and methods for face reenactment. An example method includes receiving visual data including a visible portion of a source face, determining, based on the visible portion of the source face, a first portion of source face parameters associated with a parametric face model, where the first portion corresponds to the visible portion, predicting, based partially on the visible portion of the source face, a second portion of the source face parameters, where the second portion corresponds to the rest of the source face, receiving a target video that includes a target face, determining, based on the target video, target face parameters associated with the parametric face model and corresponding to the target face, and synthesizing, using the parametric face model, based on the source face parameters and the target face parameters, an output face that includes the source face imitating a facial expression of the target face.

公开(公告)号：US11354872B2

公开(公告)日：2022-06-07

申请号：US16949700

申请日：2020-11-11

Applicant: Snap Inc.

Inventor： Roman Golobokov ,  Aleksandr Mashrabov ,  Dmitry Matov ,  Jeremy Baker Voss

IPC: G09G5/00 ,  G06T19/20 ,  G06T19/00 ,  H04L51/04 ,  H04L51/10 ,  G06T13/40 ,  G06T11/60 ,  G06N20/00 ,  G06V40/16

Abstract: The technical problem of creating an augmented reality (AR) experience that, on one hand, is accessible from a camera view user interface provided with a messaging client and that, also, can perform a modification based on a previously captured image of a user, is addressed by providing an AR component. When a user, while accessing the messaging client, engages a user selectable element representing the AR component in the camera view user interface, the messaging system loads the AR component in the messaging client. The AR component comprises a target media content object, which can be animation or live action video. The loaded AR component accesses a portrait image associated with a user and modifies the target media content using the portrait image. The resulting target media content object is displayed in the camera view user interface.

公开(公告)号：US20210368039A1

公开(公告)日：2021-11-25

申请号：US16878868

申请日：2020-05-20

Applicant: Snap Inc.

Inventor： Jeremy Voss ,  Victor Shaburov ,  Ivan Babanin ,  Aleksandr Mashrabov ,  Roman Golobokov

IPC: H04M1/725 ,  G06T11/60 ,  G06T13/80 ,  H04L12/58

Abstract: A system for customizing text messages in modifiable videos of a multimedia messaging application (MMA) is provided. In one example embodiment, the system includes a processor and a memory storing processor-executable codes, wherein the processor is configured to analyze recent messages of a user in the MMA to determine a context of the recent messages; determine, based on the context, a customized text message; select, based on the context, a list of relevant modifiable videos from a database configured to store modifiable videos, the modifiable videos being associated with preset text messages; replace the preset text messages in the relevant modifiable videos with the customized message; and render the list of relevant modifiable videos for viewing and selecting by the user, the rendering including displaying the customized text message in the relevant modifiable videos.

公开(公告)号：US11114086B2

公开(公告)日：2021-09-07

申请号：US16509370

申请日：2019-07-11

Applicant: SNAP INC.

Inventor： Pavel Savchenkov ,  Maxim Lukin ,  Aleksandr Mashrabov

IPC: G10L13/00 ,  G06K9/00 ,  G10L13/08 ,  G06T13/40

Abstract: Provided are systems and methods for text and audio-based real-time face reenactment. An example method includes receiving an input text and a target image, the target image including a target face; generating, based on the input text, a sequence of sets of acoustic features representing the input text; determining, based on the sequence of sets of acoustic features, a sequence of sets of scenario data indicating modifications of the target face for pronouncing the input text; generating, based on the sequence of sets of scenario data, a sequence of frames, wherein each of the frames includes the target face modified based on at least one of the sets of scenario data; generating, based on the sequence of frames, an output video; and synthesizing, based on the sequence of sets of acoustic features, an audio data and adding the audio data to the output video.