公开(公告)号：US11657571B2

公开(公告)日：2023-05-23

申请号：US18065555

申请日：2022-12-13

Applicant: NVIDIA Corporation

Inventor： Jon Niklas Theodor Hasselgren ,  Carl Jacob Munkberg

IPC: G06T17/20

CPC classification number: G06T17/20

Abstract: Systems and methods enable optimization of a 3D model representation comprising the shape and appearance of a particular 3D scene or object. The opaque 3D mesh (e.g., vertex positions and corresponding topology) and spatially varying material attributes are jointly optimized based on image space losses to match multiple image observations (e.g., reference images of the reference 3D scene or object). A geometric topology defines faces and/or cells in the opaque 3D mesh that are visible and may be randomly initialized and optimized through training based on the image space losses. Applying the geometry topology to an opaque 3D mesh for learning the shape improves accuracy of silhouette edges and performance compared with using transparent mesh representations. In contrast with approaches that require an initial guess for the topology and/or an exhaustive testing of possible geometric topologies, the 3D model representation is learned based on image space differences without requiring an initial guess.

公开(公告)号：US20230014245A1

公开(公告)日：2023-01-19

申请号：US17930668

申请日：2022-09-08

Applicant: NVIDIA Corporation

Inventor： Carl Jacob Munkberg ,  Jon Niklas Theodor Hasselgren ,  Anjul Patney ,  Marco Salvi ,  Aaron Eliot Lefohn ,  Donald Lee Brittain

IPC: G06T5/00 ,  G06T7/246 ,  G06T7/50

Abstract: A neural network-based rendering technique increases temporal stability and image fidelity of low sample count path tracing by optimizing a distribution of samples for rendering each image in a sequence. A sample predictor neural network learns spatio-temporal sampling strategies such as placing more samples in dis-occluded regions and tracking specular highlights. Temporal feedback enables a denoiser neural network to boost the effective input sample count and increases temporal stability. The initial uniform sampling step typically present in adaptive sampling algorithms is not needed. The sample predictor and denoiser operate at interactive rates to achieve significantly improved image quality and temporal stability compared with conventional adaptive sampling techniques.

公开(公告)号：US20220392179A1

公开(公告)日：2022-12-08

申请号：US17888207

申请日：2022-08-15

Applicant: NVIDIA Corporation

Inventor： Carl Jacob Munkberg ,  Jon Niklas Theodor Hasselgren

IPC: G06T19/20 ,  G06T7/55 ,  G06T1/20 ,  G06T15/20 ,  G06T13/20

Abstract: Appearance driven automatic three-dimensional (3D) modeling enables optimization of a 3D model comprising the shape and appearance of a particular 3D scene or object. Triangle meshes and shading models may be jointly optimized to match the appearance of a reference 3D model based on reference images of the reference 3D model. Compared with the reference 3D model, the optimized 3D model is a lower resolution 3D model that can be rendered in less time. More specifically, the optimized 3D model may include fewer geometric primitives compared with the reference 3D model. In contrast with the conventional inverse rendering or analysis-by-synthesis modeling tools, the shape and appearance representations of the 3D model are automatically generated that, when rendered, match the reference images. Appearance driven automatic 3D modeling has a number of uses, including appearance-preserving simplification of extremely complex assets, conversion between rendering systems, and even conversion between geometric scene representations.

公开(公告)号：US11475542B2

公开(公告)日：2022-10-18

申请号：US16717090

申请日：2019-12-17

Applicant: NVIDIA Corporation

Inventor： Carl Jacob Munkberg ,  Jon Niklas Theodor Hasselgren ,  Anjul Patney ,  Marco Salvi ,  Aaron Eliot Lefohn ,  Donald Lee Brittain

IPC: G06T5/00 ,  G06T7/246 ,  G06T7/50

Abstract: A neural network-based rendering technique increases temporal stability and image fidelity of low sample count path tracing by optimizing a distribution of samples for rendering each image in a sequence. A sample predictor neural network learns spatio-temporal sampling strategies such as placing more samples in dis-occluded regions and tracking specular highlights. Temporal feedback enables a denoiser neural network to boost the effective input sample count and increases temporal stability. The initial uniform sampling step typically present in adaptive sampling algorithms is not needed. The sample predictor and denoiser operate at interactive rates to achieve significantly improved image quality and temporal stability compared with conventional adaptive sampling techniques.

公开(公告)号：US10565686B2

公开(公告)日：2020-02-18

申请号：US15807401

申请日：2017-11-08

Applicant: NVIDIA Corporation

Inventor： Jaakko T. Lehtinen ,  Timo Oskari Aila ,  Jon Niklas Theodor Hasselgren ,  Carl Jacob Munkberg

IPC: G06T5/00 ,  G06N3/08 ,  G06N3/04

Abstract: A method, computer readable medium, and system are disclosed for training a neural network. The method includes the steps of selecting an input sample from a set of training data that includes input samples and noisy target samples, where the input samples and the noisy target samples each correspond to a latent, clean target sample. The input sample is processed by a neural network model to produce an output and a noisy target sample is selected from the set of training data, where the noisy target samples have a distribution relative to the latent, clean target sample. The method also includes adjusting parameter values of the neural network model to reduce differences between the output and the noisy target sample.