公开(公告)号：US20230252704A1

公开(公告)日：2023-08-10

申请号：US18136470

申请日：2023-04-19

Applicant: Snap Inc.

Inventor： Sergey Demyanov ,  Aleksei Podkin ,  Aliaksandr Siarohin ,  Aleksei Stoliar ,  Sergey Tulyakov

IPC: G06T13/00 ,  G06N3/08 ,  G06N3/045 ,  G06V40/16

CPC classification number: G06T13/00 ,  G06N3/08 ,  G06N3/045 ,  G06V40/174 ,  G06V40/171

Abstract: Systems and methods are disclosed for generating, a source image sequence using an image sensor of the computing device, the source image sequence comprising a plurality of source images depicting a head and face, identifying driving image sequence data to modify face image feature data in the source image sequence, generating, using an image transformation neural network, a modified source image sequence comprising a plurality of modified source images depicting modified versions of the head and face, and storing the modified source image sequence on the computing device.

公开(公告)号：US20230146865A1

公开(公告)日：2023-05-11

申请号：US18096338

申请日：2023-01-12

Applicant: Snap Inc.

Inventor： Enxu Yan ,  Sergey Tulyakov ,  Aleksei Podkin ,  Aleksei Stoliar

IPC: G06N3/082 ,  G06F17/16 ,  G06N3/04 ,  G06T7/10

CPC classification number: G06N3/082 ,  G06F17/16 ,  G06N3/04 ,  G06T7/10 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: A neural network pruning system can sparsely prune neural network models using an optimizer based approach that is agnostic to the model architecture being pruned. The neural network pruning system can prune by operating on the parameter vector of the full model and the gradient vector of the loss function with respect to the model parameters. The neural network pruning system can iteratively update parameters based on the gradients, while zeroing out as many parameters as possible based a preconfigured penalty.42188

公开(公告)号：US11645798B1

公开(公告)日：2023-05-09

申请号：US17303537

申请日：2021-06-01

Applicant: Snap Inc.

Inventor： Sergey Demyanov ,  Aleksei Podkin ,  Aliaksandr Siarohin ,  Aleksei Stoliar ,  Sergey Tulyakov

IPC: G06T13/00 ,  G06N3/04 ,  G06N3/08 ,  G06V40/16

CPC classification number: G06T13/00 ,  G06N3/0454 ,  G06N3/08 ,  G06V40/171 ,  G06V40/174

Abstract: Systems and methods are disclosed for generating, a source image sequence using an image sensor of the computing device, the source image sequence comprising a plurality of source images depicting a head and face, identifying driving image sequence data to modify face image feature data in the source image sequence, generating, using an image transformation neural network, a modified source image sequence comprising a plurality of modified source images depicting modified versions of the head and face, and storing the modified source image sequence on the computing device.

公开(公告)号：US20220207355A1

公开(公告)日：2022-06-30

申请号：US17318658

申请日：2021-05-12

Applicant: Snap Inc.

Inventor： Sergey Demyanov ,  Konstantin Gudkov ,  Aleksei Stoliar ,  Roman Ushakov ,  Fedor Zhdanov

IPC: G06N3/08 ,  G06N3/04 ,  G06T11/60

Abstract: Systems and methods herein describe an image manipulation system for generating modified images using a generative adversarial network. The image manipulation system accesses a pre-trained generative adversarial network (GAN), fine-tunes the pre-trained GAN by training a portion of existing neural network layers of the pre-trained GAN and newly added layers of the pre-trained GAN on a secondary image domain, adjusts the weights of the fine-tuned GAN using the weights of the pre-trained GAN, and stores the fine-tuned GAN. An image transformation system uses the generated modified images to train a subsequent neural network, which can access a face from a client device and transform it to a domain of images used for GAN fine-tuning.

公开(公告)号：US11044217B2

公开(公告)日：2021-06-22

申请号：US16703559

申请日：2019-12-04

Applicant: Snap Inc.

Inventor： Grygoriy Kozhemiak ,  Oleksandr Pyshchenko ,  Victor Shaburov ,  Trevor Stephenson ,  Aleksei Stoliar

IPC: G06F15/16 ,  H04L12/58 ,  H04L29/08

Abstract: Systems and methods are provided for receiving a first media content item associated with a first interactive object of an interactive message, receiving a second media content item associated with a second interactive object of the interactive message, generating a third media content item based on the first media content item and second media content item, wherein the third media content item comprises combined features of the first media content item and the second media content item, and causing display of the generated third media content item.

公开(公告)号：US10963748B1

公开(公告)日：2021-03-30

申请号：US16119956

申请日：2018-08-31

Applicant: Snap Inc.

Inventor： Sergey Tulyakov ,  Sergei Korolev ,  Aleksei Stoliar ,  Maksim Gusarov ,  Sergei Kotcur ,  Christopher Yale Crutchfield ,  Andrew Wan

IPC: G06K9/62 ,  G06N3/08 ,  G06N3/04

Abstract: A compact generative neural network can be distilled from a teacher generative neural network using a training network. The compact network can be trained on the input data and output data of the teacher network. The training network train the student network using a discrimination layer and one or more types of losses, such as perception loss and adversarial loss.

公开(公告)号：US20250054201A1

公开(公告)日：2025-02-13

申请号：US18231886

申请日：2023-08-09

Applicant: Snap Inc.

Inventor： Ekaterina Deyneka ,  Andrey Alejandrovich Gomez Zharkov ,  Sergey Tulyakov ,  Aleksei Stoliar ,  Konstantin Gudkov

IPC: G06T11/00 ,  G06V10/774 ,  G06V10/776

Abstract: Methods and systems are disclosed for enhancing or modifying an image by a machine learning model. The methods and systems receive an image depicting a real-world object. The methods and systems analyze the image using a machine learning model to generate a modified image that depicts one or more augmented reality stylizations overlaid on the real-world object, the machine learning model trained in multiple stages having different training data sets and different conditions applied in each of the multiple stages. The methods and systems present the modified image on a device.

公开(公告)号：US12165244B2

公开(公告)日：2024-12-10

申请号：US17974400

申请日：2022-10-26

Applicant: Snap Inc.

Inventor： Olha Rykhliuk ,  Jonathan Solichin ,  Aleksei Stoliar

IPC: G06T11/60 ,  G06F18/24 ,  G06V10/26 ,  G06V10/46

Abstract: Systems and methods herein describe receiving an image via an image capture device, using a machine learning model, generating an image augmentation decision, accessing an augmented reality content item, associating the generated image augmentation decision with the augmented reality content item, modifying the received image using the augmented reality content item and the associated image augmentation decision, and causing presentation of the modified image on a graphical user interface of a computing device.

公开(公告)号：US11900565B2

公开(公告)日：2024-02-13

申请号：US18116682

申请日：2023-03-02

Applicant: Snap Inc.

Inventor： Sergey Demyanov ,  Aleksei Podkin ,  Aleksei Stoliar ,  Vadim Velicodnii ,  Fedor Zhdanov

IPC: G06T5/00 ,  G06T5/10 ,  H04L51/10

CPC classification number: G06T5/001 ,  G06T5/10 ,  G06T2200/16 ,  G06T2207/10004 ,  G06T2207/20048 ,  G06T2207/20081 ,  G06T2207/30196 ,  H04L51/10

Abstract: A data item is identified on a device. A neural network that includes an adversarial transformation subnetwork is applied to the data item to generate a modified data item. Output indicative of the modified data item is caused to be presented on the device. The neural network further comprises an encoder and a decoder. The neural network is trained in at least two stages. At least one of the encoder and the decoder is trained in a first stage and the adversarial transformation subnetwork is trained in a second stage.

公开(公告)号：US20230334327A1

公开(公告)日：2023-10-19

申请号：US18213145

申请日：2023-06-22

Applicant: Snap Inc.

Inventor： Sergey Tulyakov ,  Sergei Korolev ,  Aleksei Stoliar ,  Maksim Gusarov ,  Sergei Kotcur ,  Christopher Yale Crutchfield ,  Andrew Wan

IPC: G06N3/088 ,  G06N3/08 ,  G06F18/21 ,  G06F18/214 ,  G06N3/045 ,  G06V10/764 ,  G06V10/774 ,  G06V10/778 ,  G06V10/82

CPC classification number: G06N3/088 ,  G06N3/08 ,  G06F18/2185 ,  G06F18/2148 ,  G06N3/045 ,  G06V10/764 ,  G06V10/7747 ,  G06V10/7788 ,  G06V10/82

Abstract: A compact generative neural network can be distilled from a teacher generative neural network using a training network. The compact network can be trained on the input data and output data of the teacher network. The training network train the student network using a discrimination layer and one or more types of losses, such as perception loss and adversarial loss.