公开(公告)号：US20240169652A1

公开(公告)日：2024-05-23

申请号：US18497945

申请日：2023-10-30

Applicant: NVIDIA CORPORATION

Inventor： Yang FU ,  Sifei LIU ,  Jan KAUTZ ,  Xueting LI ,  Shalini DE MELLO ,  Amey KULKARNI ,  Milind NAPHADE

IPC: G06T15/04 ,  G06T7/40 ,  G06T7/60 ,  G06T15/08 ,  G06T15/10

CPC classification number: G06T15/04 ,  G06T7/40 ,  G06T7/60 ,  G06T15/08 ,  G06T15/10 ,  G06T2207/10024 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: In various embodiments, a scene reconstruction model generates three-dimensional (3D) representations of scenes. The scene reconstruction model computes a first 3D feature grid based on a set of red, blue, green, and depth (RGBD) images associated with a first scene. The scene reconstruction model maps the first 3D feature grid to a first 3D representation of the first scene. The scene reconstruction model computes a first reconstruction loss based on the first 3D representation and the set of RGBD images. The scene reconstruction model modifies at least one of the first 3D feature grid, a first pre-trained geometry decoder, or a first pre-trained texture decoder based on the first reconstruction loss to generate a second 3D representation of the first scene.

公开(公告)号：US20240161383A1

公开(公告)日：2024-05-16

申请号：US18497940

申请日：2023-10-30

Applicant: NVIDIA CORPORATION

Inventor： Yang FU ,  Sifei LIU ,  Jan KAUTZ ,  Xueting LI ,  Shalini DE MELLO ,  Amey KULKARNI ,  Milind NAPHADE

IPC: G06T15/04 ,  G06T7/50 ,  G06T9/00

CPC classification number: G06T15/04 ,  G06T7/50 ,  G06T9/002 ,  G06T2207/10024 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221

Abstract: In various embodiments, a scene reconstruction model generates three-dimensional (3D) representations of scenes. The scene reconstruction model maps a first red, blue, green, and depth (RGBD) image associated with both a first scene and a first viewpoint to a first surface representation of at least a first portion of the first scene. The scene reconstruction model maps a second RGBD image associated with both the first scene and a second viewpoint to a second surface representation of at least a second portion of the first scene. The scene reconstruction model aggregates at least the first surface representation and the second surface representation in a 3D space to generate a first fused surface representation of the first scene. The scene reconstruction model maps the first fused surface representation of the first scene to a 3D representation of the first scene.

公开(公告)号：US20240161404A1

公开(公告)日：2024-05-16

申请号：US18497938

申请日：2023-10-30

Applicant: NVIDIA CORPORATION

Inventor： Yang FU ,  Sifei LIU ,  Jan KAUTZ ,  Xueting LI ,  Shalini DE MELLO ,  Amey KULKARNI ,  Milind NAPHADE

IPC: G06T17/20

CPC classification number: G06T17/20

Abstract: In various embodiments, a training application trains a machine learning model to generate three-dimensional (3D) representations of two-dimensional images. The training application maps a depth image and a viewpoint to signed distance function (SDF) values associated with 3D query points. The training application maps a red, blue, and green (RGB) image to radiance values associated with the 3DI query points. The training application computes a red, blue, green, and depth (RGBD) reconstruction loss based on at least the SDF values and the radiance values. The training application modifies at least one of a pre-trained geometry encoder, a pre-trained geometry decoder, an untrained texture encoder, or an untrained texture decoder based on the RGBD reconstruction loss to generate a trained machine learning model that generates 3D representations of RGBD images.