公开(公告)号：US20220101122A1

公开(公告)日：2022-03-31

申请号：US17357728

申请日：2021-06-24

Applicant: NVIDIA CORPORATION

Inventor： Arash VAHDAT ,  Karsten KREIS ,  Zhisheng XIAO ,  Jan KAUTZ

IPC: G06N3/08 ,  G06N3/04 ,  G06K9/00

Abstract: One embodiment of the present invention sets forth a technique for generating data using a generative model. The technique includes sampling from one or more distributions of one or more variables to generate a first set of values for the one or more variables, where the one or more distributions are used during operation of one or more portions of the generative model. The technique also includes applying one or more energy values generated via an energy-based model to the first set of values to produce a second set of values for the one or more variables. The technique further includes either outputting the set of second values as output data or performing one or more operations based on the second set of values to generate output data.

公开(公告)号：US20210397945A1

公开(公告)日：2021-12-23

申请号：US17089492

申请日：2020-11-04

Applicant: NVIDIA CORPORATION

Inventor： Arash VAHDAT ,  Jan KAUTZ

IPC: G06N3/08 ,  G06N3/04

Abstract: One embodiment of the present invention sets forth a technique for performing machine learning. The technique includes inputting a training dataset into a variational autoencoder (VAE) comprising an encoder network, a prior network, and a decoder network. The technique also includes training the VAE by updating one or more parameters of the VAE based on a smoothness of one or more outputs produced by the VAE from the training dataset. The technique further includes producing generative output that reflects a first distribution of the training dataset by applying the decoder network to one or more values sampled from a second distribution of latent variables generated by the prior network.

公开(公告)号：US20250157114A1

公开(公告)日：2025-05-15

申请号：US18623745

申请日：2024-04-01

Applicant: NVIDIA Corporation

Inventor： Ye YUAN ,  Xueting LI ,  Umar IQBAL ,  Koki NAGANO ,  Shalini DE MELLO ,  Jan KAUTZ

IPC: G06T13/40 ,  G06T17/20

Abstract: In various examples, systems and methods are disclosed relating to generating animatable characters or avatars. The system can assign a plurality of first elements of a three-dimensional (3D) model of a subject to a plurality of locations on a surface of the subject in an initial pose. Further, the system can assign a plurality of second elements to the plurality of first elements, each second element of the plurality of second elements having an opacity corresponding to a distance between the second element and the surface of the subject. Further, the system can update the plurality of second elements based at least on a target pose for the subject and one or more attributes of the subject to determine a plurality of updated second elements. Further, the system can render a representation of the subject based at least on the plurality of updated second elements.

公开(公告)号：US20250103906A1

公开(公告)日：2025-03-27

申请号：US18471196

申请日：2023-09-20

Applicant: NVIDIA CORPORATION

Inventor： Wonmin BYEON ,  Sudarshan BABU ,  Shalini DE MELLO ,  Jan KAUTZ

IPC: G06N3/0985 ,  G06N3/0895

Abstract: One embodiment of the present invention sets forth a technique for performing meta-learning. The technique includes performing a first set of training iterations to convert a prediction learning network into a first trained prediction learning network based on a first support set of training data and executing a representation learning network and the first trained prediction learning network to generate a first set of supervised training output and a first set of self-supervised training output based on a first query set of training data corresponding to the first support set of training data. The technique also includes performing a first training iteration to convert the representation learning network into a first trained representation learning network based on a first loss associated with the first set of supervised training output and a second loss associated with the first set of self-supervised training output.

公开(公告)号：US20250095350A1

公开(公告)日：2025-03-20

申请号：US18471209

申请日：2023-09-20

Applicant: NVIDIA CORPORATION

Inventor： Wonmin BYEON ,  Sudarshan BABU ,  Shalini DE MELLO ,  Jan KAUTZ

IPC: G06V10/82 ,  G06V10/776

Abstract: One embodiment of the present invention sets forth a technique for executing a machine learning model. The technique includes performing a first set of training iterations to convert a prediction learning network into a first trained prediction learning network based on a first support set associated with a first set of classes. The technique also includes executing a first trained representation learning network to convert a first data sample into a first latent representation, where the first trained representation learning network is generated by training a representation learning network using a first query set, a first set of self-supervised losses, and a first set of supervised losses. The technique further includes executing the first trained prediction learning network to convert the first latent representation into a first prediction of a first class that is not included in the second set of classes.

公开(公告)号：US20250094813A1

公开(公告)日：2025-03-20

申请号：US18471204

申请日：2023-09-20

Applicant: NVIDIA CORPORATION

Inventor： Wonmin BYEON ,  Sudarshan BABU ,  Shalini DE MELLO ,  Jan KAUTZ

IPC: G06N3/0895

Abstract: One embodiment of the present invention sets forth a technique for training a transformer neural network. The technique includes inputting a first task token and a first set of samples into the transformer neural network and training the transformer neural network using a first set of losses between predictions generated by the transformer neural network from the first task token and first set of samples as well as a first set of labels. The technique also includes converting the first task token into a second task token that is larger than the first task token, inputting the second task token and a second set of samples into the transformer neural network, and training the transformer neural network using a second set of losses between predictions generated by the transformer neural network from the second task token and the second set of samples as well as a second set of labels.

公开(公告)号：US20240251171A1

公开(公告)日：2024-07-25

申请号：US18406006

申请日：2024-01-05

Applicant: NVIDIA CORPORATION

Inventor： Iuri FROSIO ,  Yazhou XING ,  Chao LIU ,  Anjul PATNEY ,  Hongxu YIN ,  Amrita MAZUMDAR ,  Jan KAUTZ

IPC: H04N23/73 ,  H04N23/95

CPC classification number: H04N23/73 ,  H04N23/95

Abstract: One or more embodiments include receiving one or more frames of a live video captured by a video capturing device, wherein the one or more frames include a current frame that is most-recently captured, identifying a set of reference frames included in the one or more frames based on at least the current frame, wherein each frame in the set of reference frames has a different exposure level relative to the current frame, determining, using one or more neural networks, a set of missing details for one or more regions of the current frame based on the set of reference frames, generating an updated version of the current frame based on the set of details, and outputting the updated version of the current frame in real-time.

公开(公告)号：US20240169652A1

公开(公告)日：2024-05-23

申请号：US18497945

申请日：2023-10-30

Applicant: NVIDIA CORPORATION

Inventor： Yang FU ,  Sifei LIU ,  Jan KAUTZ ,  Xueting LI ,  Shalini DE MELLO ,  Amey KULKARNI ,  Milind NAPHADE

IPC: G06T15/04 ,  G06T7/40 ,  G06T7/60 ,  G06T15/08 ,  G06T15/10

CPC classification number: G06T15/04 ,  G06T7/40 ,  G06T7/60 ,  G06T15/08 ,  G06T15/10 ,  G06T2207/10024 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: In various embodiments, a scene reconstruction model generates three-dimensional (3D) representations of scenes. The scene reconstruction model computes a first 3D feature grid based on a set of red, blue, green, and depth (RGBD) images associated with a first scene. The scene reconstruction model maps the first 3D feature grid to a first 3D representation of the first scene. The scene reconstruction model computes a first reconstruction loss based on the first 3D representation and the set of RGBD images. The scene reconstruction model modifies at least one of the first 3D feature grid, a first pre-trained geometry decoder, or a first pre-trained texture decoder based on the first reconstruction loss to generate a second 3D representation of the first scene.

公开(公告)号：US20240153188A1

公开(公告)日：2024-05-09

申请号：US18455084

申请日：2023-08-24

Applicant: NVIDIA Corporation

Inventor： Jingbo WANG ,  Ye YUAN ,  Cheng XIE ,  Sanja FIDLER ,  Jan KAUTZ ,  Umar IQBAL ,  Zan GOJCIC ,  Sameh KHAMIS

IPC: G06T13/40 ,  G06T7/246

CPC classification number: G06T13/40 ,  G06T7/251 ,  G06T2207/20084 ,  G06T2210/21

Abstract: In various examples, systems and methods are disclosed relating to generating physics-plausible whole body motion, including determining a mesh sequence corresponding to a motion of at least one dynamic character of one or more dynamic characters and a mesh of a terrain using a video sequence, determining using a generative model and based at least one the mesh sequence and the mesh of the terrain, an occlusion-free motion of the at least one dynamic character by infilling physics-plausible character motions in the mesh sequence for at least one frame of the video sequence that includes an occlusion of at least a portion of the at least one dynamic character, and determining physics-plausible whole body motion of the at least one dynamic character by applying physics-based imitation upon the occlusion-free motion.

公开(公告)号：US20230267306A1

公开(公告)日：2023-08-24

申请号：US17933806

申请日：2022-09-20

Applicant: NVIDIA CORPORATION

Inventor： Benjamin ECKART ,  Jan KAUTZ ,  Chao LIU ,  Benjamin WU

IPC: G06N3/04 ,  G06T5/10

CPC classification number: G06N3/0454 ,  G06T5/10 ,  G06T2207/20056

Abstract: In various embodiments, a training application generates a trained machine learning model that represents items in a spectral domain. The training application executes a first neural network on a first set of data points associated with both a first item and the spectral domain to generate a second neural network. Subsequently, the training application generates a set of predicted data points that are associated with both the first item and the spectral domain via the second neural network. The training application generates the trained machine learning model based on the first neural network, the second neural network, and the set of predicted data points. The trained machine learning model maps one or more positions within the spectral domain to one or more values associated with an item based on a set of data points associated with both the item and the spectral domain.