公开(公告)号：US20240296623A1

公开(公告)日：2024-09-05

申请号：US18169825

申请日：2023-02-15

申请人： Nvidia Corporation

发明人： Jiahui Huang ,  Francis Williams ,  Zan Gojcic ,  Matan Atzmon ,  Or Litany ,  Sanja Fidler

IPC分类号： G06T17/20 ,  G06T15/08

CPC分类号： G06T17/20 ,  G06T15/08 ,  G06T2210/56

摘要： Approaches presented herein provide for the reconstruction of implicit multi-dimensional shapes. In one embodiment, oriented point cloud data representative of an object can be obtained using a physical scanning process. The point cloud data can be provided as input to a trained density model that can infer density functions for various points. The points can be mapped to a voxel hierarchy, allowing density functions to be determined for those voxels at the various levels that are associated with at least one point of the input point cloud. Contribution weights can be determined for the various density functions for the sparse voxel hierarchy, and the weighted density functions combined to obtain a density field. The density field can be evaluated to generate a geometric mesh where points having a zero, or near-zero, value are determined to contribute to the surface of the object.

公开(公告)号：US20240312122A1

公开(公告)日：2024-09-19

申请号：US18184459

申请日：2023-03-15

申请人： Nvidia Corporation

发明人： Nicolas Moenne-Loccoz ,  Zan Gojcic ,  Gavriel State ,  Zian Wang ,  Ignacio Llamas

IPC分类号： G06T15/50 ,  G06T5/50 ,  G06V10/60 ,  G06V10/74 ,  G06V10/82

CPC分类号： G06T15/506 ,  G06T5/50 ,  G06V10/60 ,  G06V10/761 ,  G06V10/82 ,  G06T2207/20221

摘要： Approaches presented herein provide for the generation of visual content, including different types of content representations from different sources, rendered to include consistent scene illumination for the various representations. A first render pass can produce a first image including only proxies of implicit representations (e.g., NeRF objects) under scene illumination. A second render pass can produce a second image that includes a representation of the explicit scene objects, as well as the proxies of the implicit representations, under the scene illumination, which produces secondary lighting effects. The first and second images are compared to determine irradiance ratio data for the various pixel locations. A third render pass can produce a third image that includes the implicit representations, which can have relighting performed according to the irradiance ratio data to include the secondary lighting effects. The implicit and explicit objects can then be composited to produce an image with consistent scene illumination.

公开(公告)号：US20240096017A1

公开(公告)日：2024-03-21

申请号：US17895793

申请日：2022-08-25

申请人： Nvidia Corporation

发明人： Jun Gao ,  Tianchang Shen ,  Zan Gojcic ,  Wenzheng Chen ,  Zian Wang ,  Daiqing Li ,  Or Litany ,  Sanja Fidler

IPC分类号： G06T17/20

CPC分类号： G06T17/20 ,  G06T2207/10024 ,  G06T2207/20084

摘要： Apparatuses, systems, and techniques are presented to generate digital content. In at least one embodiment, one or more neural networks are used to generate one or more textured three-dimensional meshes corresponding to one or more objects based, at least in part, one or more two-dimensional images of the one or more objects.

公开(公告)号：US20240362897A1

公开(公告)日：2024-10-31

申请号：US18634134

申请日：2024-04-12

申请人： NVIDIA Corporation

发明人： Tzofi Klinghoffer ,  Jonah Philion ,  Zan Gojcic ,  Sanja Fidler ,  Or Litany ,  Wenzheng Chen ,  Jose Manuel Alvarez Lopez

IPC分类号： G06V10/774 ,  G06T7/55 ,  G06T15/20

CPC分类号： G06V10/774 ,  G06T7/55 ,  G06T15/205 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30181 ,  G06T2207/30252

摘要： In various examples, systems and methods are disclosed relating to synthetic data generation using viewpoint augmentation for autonomous and semi-autonomous systems and applications. One or more circuits can identify a set of sequential images corresponding to a first viewpoint and generate a first transformed image corresponding to a second viewpoint using a first image of the set of sequential images as input to a machine-learning model. The one or more circuits can update the machine-learning model based at least on a loss determined according to the first transformed image and a second image of the set of sequential images.

公开(公告)号：US20240005604A1

公开(公告)日：2024-01-04

申请号：US18320716

申请日：2023-05-19

申请人： Nvidia Corporation

发明人： Karsten Julian Kreis ,  Xiaohui Zeng ,  Arash Vahdat ,  Francis Williams ,  Zan Gojcic ,  Or Litany ,  Sanja Fidler

IPC分类号： G06T17/20 ,  G06V10/44

CPC分类号： G06T17/20 ,  G06V10/44

摘要： Approaches presented herein provide for the unconditional generation of novel three dimensional (3D) object shape representations, such as point clouds or meshes. In at least one embodiment, a first denoising diffusion model (DDM) can be trained to synthesize a 1D shape latent from Gaussian noise, and a second DDM can be trained to generate a set of latent points conditioned on this 1D shape latent. The shape latent and set of latent points can be provided to a decoder to generate a 3D point cloud representative of a random object from among the object classes on which the models were trained. A surface reconstruction process may be used to generate a surface mesh from this generated point cloud. Such an approach can scale to complex and/or multimodal distributions, and can be highly flexible as it can be adapted to various tasks such as multimodal voxel- or text-guided synthesis.