公开(公告)号：US20240169652A1

公开(公告)日：2024-05-23

申请号：US18497945

申请日：2023-10-30

Applicant: NVIDIA CORPORATION

Inventor： Yang FU ,  Sifei LIU ,  Jan KAUTZ ,  Xueting LI ,  Shalini DE MELLO ,  Amey KULKARNI ,  Milind NAPHADE

IPC: G06T15/04 ,  G06T7/40 ,  G06T7/60 ,  G06T15/08 ,  G06T15/10

CPC classification number: G06T15/04 ,  G06T7/40 ,  G06T7/60 ,  G06T15/08 ,  G06T15/10 ,  G06T2207/10024 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: In various embodiments, a scene reconstruction model generates three-dimensional (3D) representations of scenes. The scene reconstruction model computes a first 3D feature grid based on a set of red, blue, green, and depth (RGBD) images associated with a first scene. The scene reconstruction model maps the first 3D feature grid to a first 3D representation of the first scene. The scene reconstruction model computes a first reconstruction loss based on the first 3D representation and the set of RGBD images. The scene reconstruction model modifies at least one of the first 3D feature grid, a first pre-trained geometry decoder, or a first pre-trained texture decoder based on the first reconstruction loss to generate a second 3D representation of the first scene.

公开(公告)号：US20240161383A1

公开(公告)日：2024-05-16

申请号：US18497940

申请日：2023-10-30

Applicant: NVIDIA CORPORATION

Inventor： Yang FU ,  Sifei LIU ,  Jan KAUTZ ,  Xueting LI ,  Shalini DE MELLO ,  Amey KULKARNI ,  Milind NAPHADE

IPC: G06T15/04 ,  G06T7/50 ,  G06T9/00

CPC classification number: G06T15/04 ,  G06T7/50 ,  G06T9/002 ,  G06T2207/10024 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221

Abstract: In various embodiments, a scene reconstruction model generates three-dimensional (3D) representations of scenes. The scene reconstruction model maps a first red, blue, green, and depth (RGBD) image associated with both a first scene and a first viewpoint to a first surface representation of at least a first portion of the first scene. The scene reconstruction model maps a second RGBD image associated with both the first scene and a second viewpoint to a second surface representation of at least a second portion of the first scene. The scene reconstruction model aggregates at least the first surface representation and the second surface representation in a 3D space to generate a first fused surface representation of the first scene. The scene reconstruction model maps the first fused surface representation of the first scene to a 3D representation of the first scene.

公开(公告)号：US20220076128A1

公开(公告)日：2022-03-10

申请号：US17017597

申请日：2020-09-10

Applicant: NVIDIA CORPORATION

Inventor： Sifei LIU ,  Shalini DE MELLO ,  Varun JAMPANI ,  Jan KAUTZ

IPC: G06N3/08 ,  G06N3/04 ,  G06K9/62

Abstract: One embodiment of the present invention sets forth a technique for performing spatial propagation. The technique includes generating a first directed acyclic graph (DAG) by connecting spatially adjacent points included in a set of unstructured points via directed edges along a first direction. The technique also includes applying a first set of neural network layers to one or more images associated with the set of unstructured points to generate (i) a set of features for the set of unstructured points and (ii) a set of pairwise affinities between the spatially adjacent points connected by the directed edges. The technique further includes generating a set of labels for the set of unstructured points by propagating the set of features across the first DAG based on the set of pairwise affinities.

公开(公告)号：US20200074707A1

公开(公告)日：2020-03-05

申请号：US16201934

申请日：2018-11-27

Applicant: NVIDIA CORPORATION

Inventor： Donghoon LEE ,  Sifei LIU ,  Jinwei GU ,  Ming-Yu LIU ,  Jan KAUTZ

IPC: G06T11/60 ,  G06T3/00 ,  G06K9/72 ,  G06K9/62 ,  G06T7/30

Abstract: One embodiment of a method includes applying a first generator model to a semantic representation of an image to generate an affine transformation, where the affine transformation represents a bounding box associated with at least one region within the image. The method further includes applying a second generator model to the affine transformation and the semantic representation to generate a shape of an object. The method further includes inserting the object into the image based on the bounding box and the shape.

公开(公告)号：US20240161468A1

公开(公告)日：2024-05-16

申请号：US18453248

申请日：2023-08-21

Applicant: NVIDIA CORPORATION

Inventor： Xueting LI ,  Stanley BIRCHFIELD ,  Shalini DE MELLO ,  Sifei LIU ,  Jiaming SONG ,  Yufei YE

IPC: G06V10/774 ,  G06T5/00 ,  G06T7/11 ,  G06V10/82 ,  G06V40/10

CPC classification number: G06V10/774 ,  G06T5/002 ,  G06T5/005 ,  G06T7/11 ,  G06V10/82 ,  G06V40/11 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30196

Abstract: Techniques are disclosed herein for generating an image. The techniques include performing one or more first denoising operations based on a first machine learning model and an input image that includes a first object to generate a mask that indicates a spatial arrangement associated with a second object interacting with the first object, and performing one or more second denoising operations based on a second machine learning model, the input image, and the mask to generate an image of the second object interacting with the first object.

公开(公告)号：US20240161404A1

公开(公告)日：2024-05-16

申请号：US18497938

申请日：2023-10-30

Applicant: NVIDIA CORPORATION

Inventor： Yang FU ,  Sifei LIU ,  Jan KAUTZ ,  Xueting LI ,  Shalini DE MELLO ,  Amey KULKARNI ,  Milind NAPHADE

IPC: G06T17/20

CPC classification number: G06T17/20

Abstract: In various embodiments, a training application trains a machine learning model to generate three-dimensional (3D) representations of two-dimensional images. The training application maps a depth image and a viewpoint to signed distance function (SDF) values associated with 3D query points. The training application maps a red, blue, and green (RGB) image to radiance values associated with the 3DI query points. The training application computes a red, blue, green, and depth (RGBD) reconstruction loss based on at least the SDF values and the radiance values. The training application modifies at least one of a pre-trained geometry encoder, a pre-trained geometry decoder, an untrained texture encoder, or an untrained texture decoder based on the RGBD reconstruction loss to generate a trained machine learning model that generates 3D representations of RGBD images.

公开(公告)号：US20220335672A1

公开(公告)日：2022-10-20

申请号：US17585449

申请日：2022-01-26

Applicant: NVIDIA Corporation

Inventor： Donghoon LEE ,  Sifei LIU ,  Jinwei GU ,  Ming-Yu LIU ,  Jan KAUTZ

IPC: G06T11/60 ,  G06T3/00 ,  G06K9/62 ,  G06T7/30 ,  G06V30/262

Abstract: One embodiment of a method includes applying a first generator model to a semantic representation of an image to generate an affine transformation, where the affine transformation represents a bounding box associated with at least one region within the image. The method further includes applying a second generator model to the affine transformation and the semantic representation to generate a shape of an object. The method further includes inserting the object into the image based on the bounding box and the shape.