公开(公告)号：US11900565B2

公开(公告)日：2024-02-13

申请号：US18116682

申请日：2023-03-02

Applicant: Snap Inc.

Inventor： Sergey Demyanov ,  Aleksei Podkin ,  Aleksei Stoliar ,  Vadim Velicodnii ,  Fedor Zhdanov

IPC: G06T5/00 ,  G06T5/10 ,  H04L51/10

CPC classification number: G06T5/001 ,  G06T5/10 ,  G06T2200/16 ,  G06T2207/10004 ,  G06T2207/20048 ,  G06T2207/20081 ,  G06T2207/30196 ,  H04L51/10

Abstract: A data item is identified on a device. A neural network that includes an adversarial transformation subnetwork is applied to the data item to generate a modified data item. Output indicative of the modified data item is caused to be presented on the device. The neural network further comprises an encoder and a decoder. The neural network is trained in at least two stages. At least one of the encoder and the decoder is trained in a first stage and the adversarial transformation subnetwork is trained in a second stage.

公开(公告)号：US20230419599A1

公开(公告)日：2023-12-28

申请号：US17846918

申请日：2022-06-22

Applicant: Snap Inc.

Inventor： Menglei Chai ,  Sergey Demyanov ,  Yunqing Hu ,  Istvan Marton ,  Daniil Ostashev ,  Aleksei Podkin

IPC: G06T15/50 ,  G06T15/80 ,  G06V10/82 ,  G06V10/774

CPC classification number: G06T15/506 ,  G06T15/80 ,  G06V10/82 ,  G06V10/774 ,  G06T2200/08 ,  G06T2200/04

Abstract: A method for applying lighting conditions to a virtual object in an augmented reality (AR) device is described. In one aspect, the method includes generating, using a camera of a mobile device, an image, accessing a virtual object corresponding to an object in the image, identifying lighting parameters of the virtual object based on a machine learning model that is pre-trained with a paired dataset, the paired dataset includes synthetic source data and synthetic target data, the synthetic source data includes environment maps and 3D scans of items depicted in the environment map, the synthetic target data includes a synthetic sphere image rendered in the same environment map, applying the lighting parameters to the virtual object, and displaying, in a display of the mobile device, the shaded virtual object as a layer to the image.

公开(公告)号：US20230206398A1

公开(公告)日：2023-06-29

申请号：US18116682

申请日：2023-03-02

Applicant: Snap Inc.

Inventor： Sergey Demyanov ,  Aleksei Podkin ,  Aleksei Stoliar ,  Vadim Velicodnii ,  Fedor Zhdanov

IPC: G06T5/00 ,  G06T5/10

CPC classification number: G06T5/001 ,  G06T5/10 ,  G06T2207/20081 ,  H04L51/10

Abstract: A data item is identified on a device. A neural network that includes an adversarial transformation subnetwork is applied to the data item to generate a modified data item. Output indicative of the modified data item is caused to be presented on the device. The neural network further comprises an encoder and a decoder. The neural network is trained in at least two stages. At least one of the encoder and the decoder is trained in a first stage and the adversarial transformation subnetwork is trained in a second stage.

公开(公告)号：US20230410438A1

公开(公告)日：2023-12-21

申请号：US17843573

申请日：2022-06-17

Applicant: Snap Inc.

Inventor： Sergey Demyanov ,  Yunqing Hu ,  Istvan Marton ,  Daniil Ostashev ,  Aleksei Podkin

IPC: G06T19/00 ,  G06T5/00 ,  G06V10/82 ,  G06V20/20 ,  G06V10/774 ,  G06T5/20

CPC classification number: G06T19/006 ,  G06T5/002 ,  G06V10/82 ,  G06V20/20 ,  G06V10/774 ,  G06T5/20 ,  G06T2207/20084 ,  G06T2207/20081 ,  G06T2207/30201

Abstract: Systems and embodiments herein describe an augmented reality (AR) object rendering system. The AR object rendering system receives an image, generates a set of noise parameters and a set of blur parameters for the image using a neural network trained on a paired dataset of images, identifies an AR object associated with the image, modifies the AR object using the set of noise parameters and the set of blur parameters, displays the modified augmented reality object within the image.

公开(公告)号：US11736717B2

公开(公告)日：2023-08-22

申请号：US17490277

申请日：2021-09-30

Applicant: Snap Inc.

Inventor： Sergey Demyanov ,  Andrew Cheng-min Lin ,  Walton Lin ,  Aleksei Podkin ,  Aleksei Stoliar ,  Sergey Tulyakov

IPC: H04N19/54 ,  H04N19/137 ,  H04N19/149 ,  H04N19/172 ,  H04L65/70 ,  G06N3/045

CPC classification number: H04N19/54 ,  G06N3/045 ,  H04L65/70 ,  H04N19/137 ,  H04N19/149 ,  H04N19/172

Abstract: Systems and methods herein describe a video compression system. The described systems and methods accesses a sequence of image frames from a first computing device, the sequence of image frames comprising a first image frame and a second image frame, detects a first set of keypoints for the first image frame, transmits the first image frame and the first set of keypoints to a second computing device, detects a second set of keypoints for the second image frame, transmits the second set of keypoints to the second computing device, causes an animated image to be displayed on the second computing device.

公开(公告)号：US11657479B2

公开(公告)日：2023-05-23

申请号：US17445362

申请日：2021-08-18

Applicant: Snap Inc.

Inventor： Sergey Demyanov ,  Aleksei Podkin ,  Aleksei Stoliar ,  Vadim Velicodnii ,  Fedor Zhdanov

IPC: G06T5/00 ,  G06T5/10 ,  H04L51/10

CPC classification number: G06T5/001 ,  G06T5/10 ,  G06T2200/16 ,  G06T2207/10004 ,  G06T2207/20048 ,  G06T2207/20081 ,  G06T2207/30196 ,  H04L51/10

Abstract: A mobile device can implement a neural network-based domain transfer scheme to modify an image in a first domain appearance to a second domain appearance. The domain transfer scheme can be configured to detect an object in the image, apply an effect to the image, and blend the image using color space adjustments and blending schemes to generate a realistic result image. The domain transfer scheme can further be configured to efficiently execute on the constrained device by removing operational layers based on resources available on the mobile device.

公开(公告)号：US11580400B1

公开(公告)日：2023-02-14

申请号：US16586635

申请日：2019-09-27

Applicant: Snap Inc.

Inventor： Enxu Yan ,  Sergey Tulyakov ,  Aleksei Podkin ,  Aleksei Stoliar

IPC: G06N3/08 ,  G06N3/082 ,  G06N3/04 ,  G06F17/16 ,  G06T7/10

Abstract: A neural network pruning system can sparsely prune neural network models using an optimizer based approach that is agnostic to the model architecture being pruned. The neural network pruning system can prune by operating on the parameter vector of the full model and the gradient vector of the loss function with respect to the model parameters. The neural network pruning system can iteratively update parameters based on the gradients, while zeroing out as many parameters as possible based a preconfigured penalty.

公开(公告)号：US20210383509A1

公开(公告)日：2021-12-09

申请号：US17445362

申请日：2021-08-18

Applicant: Snap Inc.

Inventor： Sergey Demyanov ,  Aleksei Podkin ,  Aleksei Stoliar ,  Vadim Velicodnii ,  Fedor Zhdanov

IPC: G06T5/00 ,  G06T5/10

Abstract: A mobile device can implement a neural network-based domain transfer scheme to modify an image in a first domain appearance to a second domain appearance. The domain transfer scheme can be configured to detect an object in the image, apply an effect to the image, and blend the image using color space adjustments and blending schemes to generate a realistic result image. The domain transfer scheme can further be configured to efficiently execute on the constrained device by removing operational layers based on resources available on the mobile device.

公开(公告)号：US11120526B1

公开(公告)日：2021-09-14

申请号：US16376564

申请日：2019-04-05

Applicant: Snap Inc.

Inventor： Sergey Demyanov ,  Aleksei Podkin ,  Aleksei Stoliar ,  Vadim Velicodnii ,  Fedor Zhdanov

IPC: G06T5/00 ,  G06T5/10 ,  H04L12/58

Abstract: A mobile device can implement a neural network-based domain transfer scheme to modify an image in a first domain appearance to a second domain appearance. The domain transfer scheme can be configured to detect an object in the image, apply an effect to the image, and blend the image using color space adjustments and blending schemes to generate a realistic result image. The domain transfer scheme can further be configured to efficiently execute on the constrained device by removing operational layers based on resources available on the mobile device.