公开(公告)号：US12288277B2

公开(公告)日：2025-04-29

申请号：US17827394

申请日：2022-05-27

Applicant: NVIDIA Corporation

Inventor： Huan Ling ,  Karsten Kreis ,  Daiqing Li ,  Seung Wook Kim ,  Antonio Torralba Barriuso ,  Sanja Fidler

IPC: G06T7/10 ,  G06T11/60 ,  G06V10/774 ,  G06V10/776

Abstract: In various examples, high-precision semantic image editing for machine learning systems and applications are described. For example, a generative adversarial network (GAN) may be used to jointly model images and their semantic segmentations based on a same underlying latent code. Image editing may be achieved by using segmentation mask modifications (e.g., provided by a user, or otherwise) to optimize the latent code to be consistent with the updated segmentation, thus effectively changing the original, e.g., RGB image. To improve efficiency of the system, and to not require optimizations for each edit on each image, editing vectors may be learned in latent space that realize the edits, and that can be directly applied on other images with or without additional optimizations. As a result, a GAN in combination with the optimization approaches described herein may simultaneously allow for high precision editing in real-time with straightforward compositionality of multiple edits.

公开(公告)号：US20250061153A1

公开(公告)日：2025-02-20

申请号：US18935222

申请日：2024-11-01

Applicant: Nvidia Corporation

Inventor： Hang Chu ,  Daiqing Li ,  David Jesus Acuna Marrero ,  Amlan Kar ,  Maria Shugrina ,  Ming-Yu Liu ,  Antonio Torralba Barriuso ,  Sanja Fidler

IPC: G06F16/901 ,  G06F30/13 ,  G06F30/27 ,  G06N3/044 ,  G06N3/045 ,  G06N3/047 ,  G06N3/08 ,  G06N3/084 ,  G06N5/04 ,  G06N20/10 ,  G06N20/20 ,  G06V10/764 ,  G06V10/82 ,  G06V10/84 ,  G06V20/10

Abstract: A generative model can be used for generation of spatial layouts and graphs. Such a model can progressively grow these layouts and graphs based on local statistics, where nodes can represent spatial control points of the layout, and edges can represent segments or paths between nodes, such as may correspond to road segments. A generative model can utilize an encoder-decoder architecture where the encoder is a recurrent neural network (RNN) that encodes local incoming paths into a node and the decoder is another RNN that generates outgoing nodes and edges connecting an existing node to the newly generated nodes. Generation is done iteratively, and can finish once all nodes are visited or another end condition is satisfied. Such a model can generate layouts by additionally conditioning on a set of attributes, giving control to a user in generating the layout.

公开(公告)号：US11494976B2

公开(公告)日：2022-11-08

申请号：US17193405

申请日：2021-03-05

Applicant: Nvidia Corporation

Inventor： Wenzheng Chen ,  Yuxuan Zhang ,  Sanja Fidler ,  Huan Ling ,  Jun Gao ,  Antonio Torralba Barriuso

IPC: G06T17/00 ,  G06N3/04 ,  G06N3/08 ,  G06T15/10

Abstract: Approaches are presented for training an inverse graphics network. An image synthesis network can generate training data for an inverse graphics network. In turn, the inverse graphics network can teach the synthesis network about the physical three-dimensional (3D) controls. Such an approach can provide for accurate 3D reconstruction of objects from 2D images using the trained inverse graphics network, while requiring little annotation of the provided training data. Such an approach can extract and disentangle 3D knowledge learned by generative models by utilizing differentiable renderers, enabling a disentangled generative model to function as a controllable 3D “neural renderer,” complementing traditional graphics renderers.

公开(公告)号：US20210390778A1

公开(公告)日：2021-12-16

申请号：US16898110

申请日：2020-06-10

Applicant: Nvidia Corporation

Inventor： Seung Wook Kim ,  Sanja Fidler ,  Jonah Philion ,  Antonio Torralba Barriuso

IPC: G06T19/00 ,  G06T15/00 ,  G06N3/08 ,  G06N5/04

Abstract: Apparatuses, systems, and techniques are presented to generate a simulated environment. In at least one embodiment, one or more neural networks are used to generate a simulated environment based, at least in part, on stored information associated with objects within the simulated environment.

公开(公告)号：US20200302250A1

公开(公告)日：2020-09-24

申请号：US16825199

申请日：2020-03-20

Applicant: Nvidia Corporation

Inventor： Hang Chu ,  Daiqing Li ,  David Jesus Acuna Marrero ,  Amlan Kar ,  Maria Shugrina ,  Ming-Yu Liu ,  Antonio Torralba Barriuso ,  Sanja Fidler

IPC: G06K9/62 ,  G06N3/08 ,  G06N3/04 ,  G06N5/04 ,  G06N20/10 ,  G06N20/20

Abstract: A generative model can be used for generation of spatial layouts and graphs. Such a model can progressively grow these layouts and graphs based on local statistics, where nodes can represent spatial control points of the layout, and edges can represent segments or paths between nodes, such as may correspond to road segments. A generative model can utilize an encoder-decoder architecture where the encoder is a recurrent neural network (RNN) that encodes local incoming paths into a node and the decoder is another RNN that generates outgoing nodes and edges connecting an existing node to the newly generated nodes. Generation is done iteratively, and can finish once all nodes are visited or another end condition is satisfied. Such a model can generate layouts by additionally conditioning on a set of attributes, giving control to a user in generating the layout.