公开(公告)号：US20240161383A1

公开(公告)日：2024-05-16

申请号：US18497940

申请日：2023-10-30

Applicant: NVIDIA CORPORATION

Inventor： Yang FU ,  Sifei LIU ,  Jan KAUTZ ,  Xueting LI ,  Shalini DE MELLO ,  Amey KULKARNI ,  Milind NAPHADE

IPC: G06T15/04 ,  G06T7/50 ,  G06T9/00

CPC classification number: G06T15/04 ,  G06T7/50 ,  G06T9/002 ,  G06T2207/10024 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221

Abstract: In various embodiments, a scene reconstruction model generates three-dimensional (3D) representations of scenes. The scene reconstruction model maps a first red, blue, green, and depth (RGBD) image associated with both a first scene and a first viewpoint to a first surface representation of at least a first portion of the first scene. The scene reconstruction model maps a second RGBD image associated with both the first scene and a second viewpoint to a second surface representation of at least a second portion of the first scene. The scene reconstruction model aggregates at least the first surface representation and the second surface representation in a 3D space to generate a first fused surface representation of the first scene. The scene reconstruction model maps the first fused surface representation of the first scene to a 3D representation of the first scene.

公开(公告)号：US20250157114A1

公开(公告)日：2025-05-15

申请号：US18623745

申请日：2024-04-01

Applicant: NVIDIA Corporation

Inventor： Ye YUAN ,  Xueting LI ,  Umar IQBAL ,  Koki NAGANO ,  Shalini DE MELLO ,  Jan KAUTZ

IPC: G06T13/40 ,  G06T17/20

Abstract: In various examples, systems and methods are disclosed relating to generating animatable characters or avatars. The system can assign a plurality of first elements of a three-dimensional (3D) model of a subject to a plurality of locations on a surface of the subject in an initial pose. Further, the system can assign a plurality of second elements to the plurality of first elements, each second element of the plurality of second elements having an opacity corresponding to a distance between the second element and the surface of the subject. Further, the system can update the plurality of second elements based at least on a target pose for the subject and one or more attributes of the subject to determine a plurality of updated second elements. Further, the system can render a representation of the subject based at least on the plurality of updated second elements.

公开(公告)号：US20240169652A1

公开(公告)日：2024-05-23

申请号：US18497945

申请日：2023-10-30

Applicant: NVIDIA CORPORATION

Inventor： Yang FU ,  Sifei LIU ,  Jan KAUTZ ,  Xueting LI ,  Shalini DE MELLO ,  Amey KULKARNI ,  Milind NAPHADE

IPC: G06T15/04 ,  G06T7/40 ,  G06T7/60 ,  G06T15/08 ,  G06T15/10

CPC classification number: G06T15/04 ,  G06T7/40 ,  G06T7/60 ,  G06T15/08 ,  G06T15/10 ,  G06T2207/10024 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: In various embodiments, a scene reconstruction model generates three-dimensional (3D) representations of scenes. The scene reconstruction model computes a first 3D feature grid based on a set of red, blue, green, and depth (RGBD) images associated with a first scene. The scene reconstruction model maps the first 3D feature grid to a first 3D representation of the first scene. The scene reconstruction model computes a first reconstruction loss based on the first 3D representation and the set of RGBD images. The scene reconstruction model modifies at least one of the first 3D feature grid, a first pre-trained geometry decoder, or a first pre-trained texture decoder based on the first reconstruction loss to generate a second 3D representation of the first scene.

公开(公告)号：US20240161468A1

公开(公告)日：2024-05-16

申请号：US18453248

申请日：2023-08-21

Applicant: NVIDIA CORPORATION

Inventor： Xueting LI ,  Stanley BIRCHFIELD ,  Shalini DE MELLO ,  Sifei LIU ,  Jiaming SONG ,  Yufei YE

IPC: G06V10/774 ,  G06T5/00 ,  G06T7/11 ,  G06V10/82 ,  G06V40/10

CPC classification number: G06V10/774 ,  G06T5/002 ,  G06T5/005 ,  G06T7/11 ,  G06V10/82 ,  G06V40/11 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30196

Abstract: Techniques are disclosed herein for generating an image. The techniques include performing one or more first denoising operations based on a first machine learning model and an input image that includes a first object to generate a mask that indicates a spatial arrangement associated with a second object interacting with the first object, and performing one or more second denoising operations based on a second machine learning model, the input image, and the mask to generate an image of the second object interacting with the first object.

公开(公告)号：US20240161404A1

公开(公告)日：2024-05-16

申请号：US18497938

申请日：2023-10-30

Applicant: NVIDIA CORPORATION

Inventor： Yang FU ,  Sifei LIU ,  Jan KAUTZ ,  Xueting LI ,  Shalini DE MELLO ,  Amey KULKARNI ,  Milind NAPHADE

IPC: G06T17/20

CPC classification number: G06T17/20

Abstract: In various embodiments, a training application trains a machine learning model to generate three-dimensional (3D) representations of two-dimensional images. The training application maps a depth image and a viewpoint to signed distance function (SDF) values associated with 3D query points. The training application maps a red, blue, and green (RGB) image to radiance values associated with the 3DI query points. The training application computes a red, blue, green, and depth (RGBD) reconstruction loss based on at least the SDF values and the radiance values. The training application modifies at least one of a pre-trained geometry encoder, a pre-trained geometry decoder, an untrained texture encoder, or an untrained texture decoder based on the RGBD reconstruction loss to generate a trained machine learning model that generates 3D representations of RGBD images.