公开(公告)号：US20240054709A1

公开(公告)日：2024-02-15

申请号：US18482634

申请日：2023-10-06

Applicant: Snap Inc.

Inventor： Gurunandan Krishnan Gorumkonda ,  Hsin-Ying Lee ,  Jie Xu

IPC: G06T13/20 ,  G06N3/08 ,  G06T13/40 ,  G06T13/80 ,  G06N3/044 ,  G06N3/045

CPC classification number: G06T13/205 ,  G06N3/08 ,  G06T13/40 ,  G06T13/80 ,  G06N3/044 ,  G06N3/045 ,  G10H2210/031

Abstract: Example methods for generating an animated character in dance poses to music may include generating, by at least one processor, a music input signal based on an acoustic signal associated with the music, and receiving, by the at least one processor, a model output signal from an encoding neural network. A current generated pose data is generated using a decoding neural network, the current generated pose data being based on previous generated pose data of a previous generated pose, the music input signal, and the model output signal. An animated character is generated based on a current generated pose data; and the animated character caused to be displayed by a display device.

公开(公告)号：US20240029346A1

公开(公告)日：2024-01-25

申请号：US17814063

申请日：2022-07-21

Applicant: Snap Inc.

Inventor： Zeng Huang ,  Menglei Chai ,  Sergey Tulyakov ,  Kyle Olszewski ,  Hsin-Ying Lee

IPC: G06T17/00 ,  G06T15/04

CPC classification number: G06T17/00 ,  G06T15/04 ,  G06T2207/10028

Abstract: A system to enable 3D hair reconstruction and rendering from a single reference image which performs a multi-stage process that utilizes both a 3D implicit representation and a 2D parametric embedding space.

公开(公告)号：US20230386158A1

公开(公告)日：2023-11-30

申请号：US17814391

申请日：2022-07-22

Applicant: Snap Inc.

Inventor： Menglei Chai ,  Sergey Tulyakov ,  Jian Ren ,  Hsin-Ying Lee ,  Kyle Olszewski ,  Zeng Huang ,  Zezhou Cheng

IPC: G06T19/20 ,  G06T17/00

CPC classification number: G06T19/20 ,  G06T17/00 ,  G06T2219/2012 ,  G06T2219/2021

Abstract: Systems, computer readable media, and methods herein describe an editing system where a three-dimensional (3D) object can be edited by editing a 2D sketch or 2D RGB views of the 3D object. The editing system uses multi-modal (MM) variational auto-decoders (VADs)(MM-VADs) that are trained with a shared latent space that enables editing 3D objects by editing 2D sketches of the 3D objects. The system determines a latent code that corresponds to an edited or sketched 2D sketch. The latent code is then used to generate a 3D object using the MM-VADs with the latent code as input. The latent space is divided into a latent space for shapes and a latent space for colors. The MM-VADs are trained with variational auto-encoders (VAE) and a ground truth.

公开(公告)号：US12094073B2

公开(公告)日：2024-09-17

申请号：US17814391

申请日：2022-07-22

Applicant: Snap Inc.

Inventor： Menglei Chai ,  Sergey Tulyakov ,  Jian Ren ,  Hsin-Ying Lee ,  Kyle Olszewski ,  Zeng Huang ,  Zezhou Cheng

IPC: G06T19/20 ,  G06T9/00 ,  G06T11/00 ,  G06T17/00

CPC classification number: G06T19/20 ,  G06T17/00 ,  G06T2219/2012 ,  G06T2219/2021

Abstract: Systems, computer readable media, and methods herein describe an editing system where a three-dimensional (3D) object can be edited by editing a 2D sketch or 2D RGB views of the 3D object. The editing system uses multi-modal (MM) variational auto-decoders (VADs)(MM-VADs) that are trained with a shared latent space that enables editing 3D objects by editing 2D sketches of the 3D objects. The system determines a latent code that corresponds to an edited or sketched 2D sketch. The latent code is then used to generate a 3D object using the MM-VADs with the latent code as input. The latent space is divided into a latent space for shapes and a latent space for colors. The MM-VADs are trained with variational auto-encoders (VAE) and a ground truth.

公开(公告)号：US20250111628A1

公开(公告)日：2025-04-03

申请号：US18375332

申请日：2023-09-29

Applicant: Snap Inc.

Inventor： Songfang Han ,  Sergei Korolev ,  Hsin-Ying Lee ,  Aleksei Stoliar

IPC: G06T19/20 ,  G06T7/90

Abstract: An artificial intelligence (AI) network or neural network is trained to generate three-dimensional (3D) models or shapes with color from two-dimensional (2D) input images and input text describing the 3D model with color. Example methods include converting a first three-dimensional (3D) model from a first representation to a second representation, the second representation including color information for the 3D model and inputting the second representation into an encoder to generate a third representation having a lower dimension than the second representation. The method further includes inputting the third representation into a decoder to generate a fourth representation having a same dimension as the second representation and generating a second 3D model from the fourth representation. The method further includes determining losses between the first 3D model and the second 3D model and updating weights of the encoder and the decoder based on the losses.

公开(公告)号：US11816773B2

公开(公告)日：2023-11-14

申请号：US17487558

申请日：2021-09-28

Applicant: Snap Inc.

Inventor： Gurunandan Krishnan Gorumkonda ,  Hsin-Ying Lee ,  Jie Xu

IPC: G06T13/20 ,  G06N3/08 ,  G06T13/40 ,  G06T13/80 ,  G06N3/044 ,  G06N3/045

CPC classification number: G06T13/205 ,  G06N3/044 ,  G06N3/045 ,  G06N3/08 ,  G06T13/40 ,  G06T13/80 ,  G10H2210/031

Abstract: Example methods for generating an animated character in dance poses to music may include generating, by at least one processor, a music input signal based on an acoustic signal associated with the music, and receiving, by the at least one processor, a model output signal from an encoding neural network. A current generated pose data is generated using a decoding neural network, the current generated pose data being based on previous generated pose data of a previous generated pose, the music input signal, and the model output signal. An animated character is generated based on a current generated pose data; and the animated character caused to be displayed by a display device.