公开(公告)号：US20240153188A1

公开(公告)日：2024-05-09

申请号：US18455084

申请日：2023-08-24

Applicant: NVIDIA Corporation

Inventor： Jingbo WANG ,  Ye YUAN ,  Cheng XIE ,  Sanja FIDLER ,  Jan KAUTZ ,  Umar IQBAL ,  Zan GOJCIC ,  Sameh KHAMIS

IPC: G06T13/40 ,  G06T7/246

CPC classification number: G06T13/40 ,  G06T7/251 ,  G06T2207/20084 ,  G06T2210/21

Abstract: In various examples, systems and methods are disclosed relating to generating physics-plausible whole body motion, including determining a mesh sequence corresponding to a motion of at least one dynamic character of one or more dynamic characters and a mesh of a terrain using a video sequence, determining using a generative model and based at least one the mesh sequence and the mesh of the terrain, an occlusion-free motion of the at least one dynamic character by infilling physics-plausible character motions in the mesh sequence for at least one frame of the video sequence that includes an occlusion of at least a portion of the at least one dynamic character, and determining physics-plausible whole body motion of the at least one dynamic character by applying physics-based imitation upon the occlusion-free motion.

公开(公告)号：US20240185523A1

公开(公告)日：2024-06-06

申请号：US18339936

申请日：2023-06-22

Applicant: NVIDIA CORPORATION

Inventor： Dongsu ZHANG ,  Amlan KAR ,  Francis WILLIAMS ,  Zan GOJCIC ,  Karsten KREIS ,  Sanja FIDLER

IPC: G06T17/10

CPC classification number: G06T17/10

Abstract: In various examples, a technique for performing three-dimensional (3D) scene completion includes determining an initial representation of a first 3D scene. The technique also includes executing a machine learning model to generate a first update to the initial representation at a previous time step and a second update to the initial representation at a current time step, wherein the second update is generated based at least on a threshold applied to a set of predictions corresponding to the first update. The technique also includes generating a 3D model of the 3D scene based at least on the second update to the initial representation.

公开(公告)号：US20250086896A1

公开(公告)日：2025-03-13

申请号：US18465363

申请日：2023-09-12

Applicant: NVIDIA Corporation

Inventor： Or LITANY ,  Sanja FIDLER ,  Cho-Ying WU ,  Huan LING ,  Zan GOJCIC ,  Riccardo DE LUTIO ,  Sameh KHAMIS

IPC: G06T19/00 ,  G06T7/194 ,  G06V10/25 ,  G06V10/44 ,  G06V10/764 ,  G06V10/82

Abstract: In various examples, systems and methods are disclosed relating to neural networks for three-dimensional (3D) scene representations and modifying the 3D scene representations. In some implementations, a diffusion model can be configured to modify selected portions of 3D scenes represented using neural radiance fields, without painting back in content of the selected portions that was originally present. A first view of the neural radiance fields can be inpainted to remove a target feature from the first view, and used as guidance for updating the neural radiance field so that the target feature can be realistically removed from various second views of the neural radiance fields while context is retained outside of the selected portions.

公开(公告)号：US20250045980A1

公开(公告)日：2025-02-06

申请号：US18361987

申请日：2023-07-31

Applicant: NVIDIA Corporation

Inventor： Tianshi CAO ,  Kangxue YIN ,  Nicholas Mark Worth SHARP ,  Karsten Julian KREIS ,  Sanja FIDLER

IPC: G06T11/00 ,  G06T5/00 ,  G06T15/20

Abstract: Aspects of this technical solution can obtain, according to a plurality of cameras oriented toward the surface of a three-dimensional (3D) model having a surface including a two-dimensional (2D) texture model, input according to corresponding views from the plurality of cameras of the 2D texture model on the surface of the 3D model, and generate, according to the input and according to a model configured to generate a two-dimensional (2D) image, an output including a 2D texture for the 3D model, the output responsive to receiving an indication of the 3D model and the 2D texture.

公开(公告)号：US20240092390A1

公开(公告)日：2024-03-21

申请号：US17949991

申请日：2022-09-21

Applicant: NVIDIA Corporation

Inventor： Jonah PHILION ,  Jeevan DEVARANJAN ,  Xue Bin PENG ,  Sanja FIDLER

IPC: B60W60/00 ,  B60W30/095 ,  B60W50/00 ,  B60W50/14

CPC classification number: B60W60/0015 ,  B60W30/0953 ,  B60W50/0097 ,  B60W50/14 ,  B60W60/0011 ,  B60W2050/146

Abstract: In various examples, systems and methods are presented for model-based trajectory simulation of agents in a simulated environment. Traffic simulators mimic reality so that autonomous or semi-autonomous vehicle design teams can validate driving models in environments that have diversity and complexity. In some embodiments, for a model-controlled agent of a simulation environment, a plurality of navigation probability distributions are generated, each of the plurality of navigation probability distributions defining a candidate trajectory for the agent to follow. A trajectory is selected for the agent based at least on at least one of the plurality of navigation probability distributions, and the agent is moved within the simulation environment based at least on the selected trajectory. In some embodiments, a search algorithm may be applied across multiple time-steps of a simulation, for example, to identify the occurrence of collision-free sequences of navigation probability distributions.

公开(公告)号：US20250131700A1

公开(公告)日：2025-04-24

申请号：US18674666

申请日：2024-05-24

Applicant: NVIDIA Corporation

Inventor： Sanja FIDLER ,  Matan ATZMON ,  Jiahui HUANG ,  Or LITANY ,  Francis WILLIAMS

IPC: G06V10/82 ,  G06V10/26 ,  G06V20/58

Abstract: In various examples, a technique for modeling equivariance in point neural networks includes determining a first partition prediction associated with partitioning of a plurality of points included in a scene into a first set of parts. The technique also includes generating, using a neural network, a second partition prediction associated with partitioning of the plurality of points into a second set of parts based at least on one or more aggregations associated with the first set of parts. The technique further includes determining a plurality of piecewise equivariant regions included in the scene based on the second partition prediction and generating an object recognition result associated with the plurality of points based on the plurality of piecewise equivariant regions.

公开(公告)号：US20250131685A1

公开(公告)日：2025-04-24

申请号：US18674668

申请日：2024-05-24

Applicant: NVIDIA Corporation

Inventor： Sanja FIDLER ,  Matan Atzmon ,  Jiahui Huang ,  Or Litany ,  Francis Williams

IPC: G06V10/44 ,  G06V10/82

Abstract: In various examples, a technique for modeling equivariance in point neural networks includes generating, via execution of one or more layers included in a neural network, a set of features associated with a first partition prediction for a plurality of points included in a scene. The technique also includes applying, to the set of features, one or more transformations included in a frame associated with the plurality of points to generate a set of equivariant features. The technique further includes generating a second partition prediction for the plurality of points based at least on the set of equivariant features, and causing an object recognition result associated with the plurality of points to be generated based at least on the second partition prediction.

公开(公告)号：US20250111109A1

公开(公告)日：2025-04-03

申请号：US18664597

申请日：2024-05-15

Applicant: NVIDIA Corporation

Inventor： Jonah PHILION ,  Sanja FIDLER ,  Jason PENG

IPC: G06F30/27

Abstract: In various examples, systems and methods are disclosed relating to generating tokens for traffic modeling. One or more circuits can identify trajectories in a dataset, and generate actions from the identified trajectories. The one or more circuits can generate, based at least on the plurality of actions and at least one trajectory of the plurality of trajectories, a set of tokens representing actions to generate trajectories of one or more agents in a simulation. The one or more circuits may update a transformer model to generate simulated actions for simulated agents based at least on tokens generated from the trajectories in the dataset.

公开(公告)号：US20250054288A1

公开(公告)日：2025-02-13

申请号：US18366394

申请日：2023-08-07

Applicant: NVIDIA Corporation

Inventor： Yuan-Hong LIAO ,  David Jesus ACUNA MARRERO ,  James LUCAS ,  Rafid MAHMOOD ,  Sanja FIDLER ,  Viraj Uday PRABHU

IPC: G06V10/82 ,  G06V20/70

Abstract: Various examples relate to translating image labels from one domain (e.g., a synthetic domain) to another domain (e.g., a real-world domain) to improve model performance on real-world datasets and applications. Systems and methods are disclosed that provide an unsupervised label translator that may employ a generative adversarial network (GAN)-based approach. In contrast to conventional systems, the disclosed approach can employ a data-centric perspective that addresses systematic mismatches between datasets from different sources.

公开(公告)号：US20230377324A1

公开(公告)日：2023-11-23

申请号：US18319689

申请日：2023-05-18

Applicant: NVIDIA Corporation

Inventor： Seung Wook KIM ,  Karsten Julian KREIS ,  Daiqing LI ,  Sanja FIDLER ,  Antonio TORRALBA BARRIUSO

IPC: G06V10/82 ,  G06V10/77

CPC classification number: G06V10/82 ,  G06V10/7715

Abstract: In various examples, systems and methods are disclosed relating to multi-domain generative adversarial networks with learned warp fields. Input data can be generated according to a noise function and provided as input to a generative machine-learning model. The generative machine-learning model can determine a plurality of output images each corresponding to one of a respective plurality of image domains. The generative machine-learning model can include at least one layer to generate a plurality of morph maps each corresponding to one of the respective plurality of image domains. The output images can be presented using a display device.