公开(公告)号：US11836597B2

公开(公告)日：2023-12-05

申请号：US16397511

申请日：2019-04-29

Applicant: NVIDIA Corporation

Inventor： Anjul Patney ,  Aaron Eliot Lefohn

IPC: G06N3/045 ,  G06T5/00 ,  G06T7/00

CPC classification number: G06N3/045 ,  G06T5/002 ,  G06T7/0008

Abstract: Motivated by the ability of humans to quickly and accurately detect visual artifacts in images, a neural network model is trained to identify and locate visual artifacts in a sequence of rendered images without comparing the sequence of rendered images against a ground truth reference. Examples of visual artifacts include aliasing, blurriness, mosaicking, and overexposure. The neural network model provides a useful fully-automated tool for evaluating the quality of images produced by rendering systems. The neural network model may be trained to evaluate the quality of images for video processing, encoding, and/or compression techniques. In an embodiment, the sequence includes at least four images corresponding to a video or animation.

公开(公告)号：US20230014245A1

公开(公告)日：2023-01-19

申请号：US17930668

申请日：2022-09-08

Applicant: NVIDIA Corporation

Inventor： Carl Jacob Munkberg ,  Jon Niklas Theodor Hasselgren ,  Anjul Patney ,  Marco Salvi ,  Aaron Eliot Lefohn ,  Donald Lee Brittain

IPC: G06T5/00 ,  G06T7/246 ,  G06T7/50

Abstract: A neural network-based rendering technique increases temporal stability and image fidelity of low sample count path tracing by optimizing a distribution of samples for rendering each image in a sequence. A sample predictor neural network learns spatio-temporal sampling strategies such as placing more samples in dis-occluded regions and tracking specular highlights. Temporal feedback enables a denoiser neural network to boost the effective input sample count and increases temporal stability. The initial uniform sampling step typically present in adaptive sampling algorithms is not needed. The sample predictor and denoiser operate at interactive rates to achieve significantly improved image quality and temporal stability compared with conventional adaptive sampling techniques.

公开(公告)号：US11475542B2

公开(公告)日：2022-10-18

申请号：US16717090

申请日：2019-12-17

Applicant: NVIDIA Corporation

Inventor： Carl Jacob Munkberg ,  Jon Niklas Theodor Hasselgren ,  Anjul Patney ,  Marco Salvi ,  Aaron Eliot Lefohn ,  Donald Lee Brittain

IPC: G06T5/00 ,  G06T7/246 ,  G06T7/50

Abstract: A neural network-based rendering technique increases temporal stability and image fidelity of low sample count path tracing by optimizing a distribution of samples for rendering each image in a sequence. A sample predictor neural network learns spatio-temporal sampling strategies such as placing more samples in dis-occluded regions and tracking specular highlights. Temporal feedback enables a denoiser neural network to boost the effective input sample count and increases temporal stability. The initial uniform sampling step typically present in adaptive sampling algorithms is not needed. The sample predictor and denoiser operate at interactive rates to achieve significantly improved image quality and temporal stability compared with conventional adaptive sampling techniques.

公开(公告)号：US20220012596A1

公开(公告)日：2022-01-13

申请号：US16925085

申请日：2020-07-09

Applicant: NVIDIA Corporation

Inventor： Weili Nie ,  Tero Tapani Karras ,  Animesh Garg ,  Shoubhik Debnath ,  Anjul Patney ,  Anima Anandkumar

IPC: G06N3/08 ,  G06N3/063 ,  G06N3/04

Abstract: Apparatuses, systems, and techniques used to train one or more neural networks to generate images comprising one or more features. In at least one embodiment, one or more neural networks are trained to determine one or more styles for an input image and then generate features associated with said one or more styles in an output image.

公开(公告)号：US10573061B2

公开(公告)日：2020-02-25

申请号：US16024591

申请日：2018-06-29

Applicant: NVIDIA Corporation

Inventor： Qi Sun ,  Anjul Patney ,  Omer Shapira ,  Morgan McGuire ,  Aaron Eliot Lefohn ,  David Patrick Luebke

IPC: G06T19/00 ,  G06F3/01 ,  G06T15/20 ,  G06T19/20

Abstract: A method, computer readable medium, and system are disclosed for redirecting a user's movement through a physical space while the user views a virtual environment. A temporary visual suppression event is detected when a user's eyes move relative to the user's head while viewing a virtual scene displayed on a display device, an orientation of the virtual scene relative to the user is modified to direct the user to physically move along a planned path through a virtual environment corresponding to the virtual scene, and the virtual scene is displayed on the display device according to the modified orientation.

公开(公告)号：US20190012832A1

公开(公告)日：2019-01-10

申请号：US16024609

申请日：2018-06-29

Applicant: NVIDIA Corporation

Inventor： Qi Sun ,  Anjul Patney ,  Omer Shapira ,  Morgan McGuire ,  Aaron Eliot Lefohn ,  David Patrick Luebke

IPC: G06T19/00 ,  G06T15/00

Abstract: A method, computer readable medium, and system are disclosed for computing a path for a user to move along within a physical space while viewing a virtual environment in a virtual reality system. A path for a user to physically move along through a virtual environment is determined based on waypoints and at least one characteristic of the physical environment within which the user is positioned, position data for the user is received indicating whether and how much a current path taken by the user has deviated from the path, and an updated path is computed through the virtual environment based on the waypoints and the at least one characteristic of the physical environment.

公开(公告)号：US20190012824A1

公开(公告)日：2019-01-10

申请号：US16024591

申请日：2018-06-29

Applicant: NVIDIA Corporation

Inventor： Qi Sun ,  Anjul Patney ,  Omer Shapira ,  Morgan McGuire ,  Aaron Eliot Lefohn ,  David Patrick Luebke

IPC: G06T15/20 ,  G06F3/01 ,  G06T19/00 ,  G06T19/20

Abstract: A method, computer readable medium, and system are disclosed for redirecting a user's movement through a physical space while the user views a virtual environment. A temporary visual suppression event is detected when a user's eyes move relative to the user's head while viewing a virtual scene displayed on a display device, an orientation of the virtual scene relative to the user is modified to direct the user to physically move along a planned path through a virtual environment corresponding to the virtual scene, and the virtual scene is displayed on the display device according to the modified orientation.

公开(公告)号：US20180343436A1

公开(公告)日：2018-11-29

申请号：US16052537

申请日：2018-08-01

Applicant: NVIDIA Corporation

Inventor： Marco Salvi ,  Anjul Patney ,  Aaron Eliot Lefohn

IPC: H04N13/122 ,  G06T1/60 ,  G06T11/40 ,  H04N13/106 ,  H04N13/15 ,  H04N13/00

CPC classification number: H04N13/122 ,  G06T1/60 ,  G06T11/40 ,  H04N13/106 ,  H04N13/15 ,  H04N2013/0081

Abstract: A method, computer readable medium, and system are disclosed for image processing to reduce aliasing using a temporal anti-aliasing algorithm modified to implement variance clipping. The method includes the step of generating a current frame of image data in a memory. Then, each pixel in the current frame of image data is processed by: sampling a resolved pixel color for a corresponding pixel in a previous frame of image data stored in the memory, adjusting the resolved pixel color based on a statistical distribution of color values for a plurality of samples in the neighborhood of the pixel in the current frame of image data to generate an adjusted pixel color, and blending a color value for the pixel in the current frame of image data with the adjusted pixel color to generate a resolved pixel color for the pixel in the current frame of image data.

公开(公告)号：US20180204307A1

公开(公告)日：2018-07-19

申请号：US15874611

申请日：2018-01-18

Applicant: NVIDIA Corporation

Inventor： Christoph H. Schied ,  Marco Salvi ,  Anton S. Kaplanyan ,  Aaron Eliot Lefohn ,  John Matthew Burgess ,  Anjul Patney ,  Christopher Ryan Wyman

IPC: G06T5/00 ,  G06T5/20

CPC classification number: G06T5/20 ,  G06K9/00986 ,  G06K9/6273 ,  G06K9/66 ,  G06T1/20 ,  G06T5/002 ,  G06T11/60 ,  G06T2207/10016 ,  G06T2207/10024 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20182

Abstract: A method, computer readable medium, and system are disclosed for performing spatiotemporal filtering. The method includes the steps of applying, utilizing a processor, a temporal filter of a filtering pipeline to a current image frame, using a temporal reprojection, to obtain a color and auxiliary information for each pixel within the current image frame, providing the auxiliary information for each pixel within the current image frame to one or more subsequent filters of the filtering pipeline, and creating a reconstructed image for the current image frame, utilizing the one or more subsequent filters of the filtering pipeline.

公开(公告)号：US20170263046A1

公开(公告)日：2017-09-14

申请号：US15453822

申请日：2017-03-08

Applicant: NVIDIA Corporation

Inventor： Anjul Patney ,  Marco Salvi ,  Joohwan Kim ,  Anton S. Kaplanyan ,  Christopher Ryan Wyman ,  Nir Benty ,  David Patrick Luebke ,  Aaron Eliot Lefohn

IPC: G06T15/20 ,  G06T15/80 ,  G06T5/20 ,  G06T15/50 ,  G06T15/60 ,  G06T5/00 ,  G06T15/00 ,  G06T15/04

CPC classification number: G06T15/20 ,  G02B27/017 ,  G06F3/013 ,  G06K9/40 ,  G06T2210/36

Abstract: A method, computer readable medium, and system are disclosed for rendering images utilizing a foveated rendering algorithm with post-process filtering to enhance a contrast of the foveated image. The method includes the step of receiving a three-dimensional scene, rendering the 3D scene according to a foveated rendering algorithm to generate a foveated image, and filtering the foveated image using a contrast-enhancing filter to generate a filtered foveated image. The foveated rendering algorithm may incorporate aspects of coarse pixel shading, mipmapped texture maps, linear efficient anti-aliased normal maps, exponential variance shadow maps, and specular anti-aliasing techniques. The foveated rendering algorithm may also be combined with temporal anti-aliasing techniques to further reduce artifacts in the foveated image.