公开(公告)号：US20250165777A1

公开(公告)日：2025-05-22

申请号：US18740294

申请日：2024-06-11

Applicant: NVIDIA CORPORATION

Inventor： Michael RANZINGER ,  Gregory HEINRICH ,  Jan KAUTZ ,  Pavlo MOLCHANOV

IPC: G06N3/08 ,  G06N3/045

Abstract: One embodiment of a method for training a first machine learning model includes processing first data via a plurality of trained machine learning models to generate a plurality of first outputs, processing the first data via the first machine learning model to generate a second output, processing the second output via a plurality of projection heads to generate a plurality of third outputs, computing a plurality of losses based on the plurality of first outputs and the plurality of third outputs, and performing one or more operations to update one or more parameters of the first machine learning model and one or more parameters of the plurality of projection heads based on the plurality of losses.

公开(公告)号：US20240303840A1

公开(公告)日：2024-09-12

申请号：US18508139

申请日：2023-11-13

Applicant: NVIDIA CORPORATION

Inventor： Chao LIU ,  Benjamin ECKART ,  Jan KAUTZ

IPC: G06T7/50 ,  G06T7/20 ,  G06V10/762

CPC classification number: G06T7/50 ,  G06T7/20 ,  G06V10/762

Abstract: The disclosed method for generating a first depth map for a first frame of a video includes performing one or more operations to generate a first intermediate depth map based on the first frame and a second frame preceding the first frame within the video, performing one or more operations to generate a second intermediate depth map based on the first frame, and performing one or more operations to combine the first intermediate depth map and the second intermediate depth map to generate the first depth map.

公开(公告)号：US20230316458A1

公开(公告)日：2023-10-05

申请号：US18173589

申请日：2023-02-23

Applicant: NVIDIA Corporation

Inventor： Yuzhuo REN ,  Kenneth TURKOWSKI ,  Nuri Murat ARAR ,  Orazio GALLO ,  Jan KAUTZ ,  Niranjan AVADHANAM ,  Hang SU

IPC: G06T3/40 ,  G06T7/73

CPC classification number: G06T3/4038 ,  G06T7/74

Abstract: In various examples, dynamic seam placement is used to position seams in regions of overlapping image data to avoid crossing salient objects or regions. Objects may be detected from image frames representing overlapping views of an environment surrounding an ego-object such as a vehicle. The images may be aligned to create an aligned composite image or surface (e.g., a panorama, a 360° image, bowl shaped surface) with regions of overlapping image data, and a representation of the detected objects and/or salient regions (e.g., a saliency mask) may be generated and projected onto the aligned composite image or surface. Seams may be positioned in the overlapping regions to avoid or minimize crossing salient pixels represented in the projected masks, and the image data may be blended at the seams to create a stitched image or surface (e.g., a stitched panorama, stitched 360° image, stitched textured surface).

公开(公告)号：US20230186077A1

公开(公告)日：2023-06-15

申请号：US17841577

申请日：2022-06-15

Applicant: NVIDIA CORPORATION

Inventor： Hongxu YIN ,  Jan KAUTZ ,  Jose Manuel ALVAREZ LOPEZ ,  Arun MALLYA ,  Pavlo MOLCHANOV ,  Arash VAHDAT

IPC: G06N3/08 ,  G06N3/04

CPC classification number: G06N3/08 ,  G06N3/0481

Abstract: One embodiment of the present invention sets forth a technique for executing a transformer neural network. The technique includes computing a first set of halting scores for a first set of tokens that has been input into a first layer of the transformer neural network. The technique also includes determining that a first halting score included in the first set of halting scores exceeds a threshold value. The technique further includes in response to the first halting score exceeding the threshold value, causing a first token that is included in the first set of tokens and is associated with the first halting score not to be processed by one or more layers within the transformer neural network that are subsequent to the first layer.

公开(公告)号：US20240161383A1

公开(公告)日：2024-05-16

申请号：US18497940

申请日：2023-10-30

Applicant: NVIDIA CORPORATION

Inventor： Yang FU ,  Sifei LIU ,  Jan KAUTZ ,  Xueting LI ,  Shalini DE MELLO ,  Amey KULKARNI ,  Milind NAPHADE

IPC: G06T15/04 ,  G06T7/50 ,  G06T9/00

CPC classification number: G06T15/04 ,  G06T7/50 ,  G06T9/002 ,  G06T2207/10024 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221

Abstract: In various embodiments, a scene reconstruction model generates three-dimensional (3D) representations of scenes. The scene reconstruction model maps a first red, blue, green, and depth (RGBD) image associated with both a first scene and a first viewpoint to a first surface representation of at least a first portion of the first scene. The scene reconstruction model maps a second RGBD image associated with both the first scene and a second viewpoint to a second surface representation of at least a second portion of the first scene. The scene reconstruction model aggregates at least the first surface representation and the second surface representation in a 3D space to generate a first fused surface representation of the first scene. The scene reconstruction model maps the first fused surface representation of the first scene to a 3D representation of the first scene.

公开(公告)号：US20240119361A1

公开(公告)日：2024-04-11

申请号：US18348286

申请日：2023-07-06

Applicant: NVIDIA CORPORATION

Inventor： Hongxu YIN ,  Wonmin BYEON ,  Jan KAUTZ ,  Divyam MADAAN ,  Pavlo MOLCHANOV

IPC: G06N20/00

CPC classification number: G06N20/00

Abstract: One embodiment of a method for training a first machine learning model having a different architecture than a second machine learning model includes receiving a first data set, performing one or more operations to generate a second data set based on the first data set and the second machine learning model, wherein the second data set includes at least one feature associated with one or more tasks that the second machine learning model was previously trained to perform, and performing one or more operations to train the first machine learning model based on the second data set and the second machine learning model.

公开(公告)号：US20220101145A1

公开(公告)日：2022-03-31

申请号：US17357738

申请日：2021-06-24

Applicant: NVIDIA CORPORATION

Inventor： Arash VAHDAT ,  Karsten KREIS ,  Zhisheng XIAO ,  Jan KAUTZ

IPC: G06N3/08 ,  G06N3/04

Abstract: One embodiment sets forth a technique for creating a generative model. The technique includes generating a trained generative model with a first component that converts data points in the training dataset into latent variable values, a second component that learns a distribution of the latent variable values, and a third component that converts the latent variable values into output distributions. The technique also includes training an energy-based model to learn an energy function based on values sampled from a first distribution associated with the training dataset and values sampled from a second distribution during operation of the trained generative model. The technique further includes creating a joint model that includes one or more portions of the trained generative model and the energy-based model, and that applies energy values from the energy-based model to samples from the second distribution to produce additional values used to generate a new data point.

公开(公告)号：US20210241489A1

公开(公告)日：2021-08-05

申请号：US17237728

申请日：2021-04-22

Applicant: NVIDIA Corporation

Inventor： Xiaodong YANG ,  Ming-Yu LIU ,  Jan KAUTZ ,  Fanyi XIAO ,  Xitong YANG

IPC: G06T7/73 ,  G06K9/00 ,  G06T7/277

Abstract: Iterative prediction systems and methods for the task of action detection process an inputted sequence of video frames to generate an output of both action tubes and respective action labels, wherein the action tubes comprise a sequence of bounding boxes on each video frame. An iterative predictor processes large offsets between the bounding boxes and the ground-truth.

公开(公告)号：US20230267656A1

公开(公告)日：2023-08-24

申请号：US17933813

申请日：2022-09-20

Applicant: NVIDIA CORPORATION

Inventor： Benjamin ECKART ,  Jan KAUTZ ,  Chao LIU ,  Benjamin WU

IPC: G06T11/00 ,  G06T7/00

CPC classification number: G06T11/003 ,  G06T7/0012 ,  G06T2207/20056 ,  G06T2207/20081

Abstract: In various embodiments, an inference application constructs medical images. The inference application executes a first trained machine learning model on a set of data points associated with a both a medical item and a spectral domain to generate a second model that represents the medical item within the spectral domain. The inference application maps a set of positions to a set of predicted values associated with both the medical item and the spectral domain via the second model. The inference application constructs an image of the medical item based on the first set of predicted values.

公开(公告)号：US20220076128A1

公开(公告)日：2022-03-10

申请号：US17017597

申请日：2020-09-10

Applicant: NVIDIA CORPORATION

Inventor： Sifei LIU ,  Shalini DE MELLO ,  Varun JAMPANI ,  Jan KAUTZ

IPC: G06N3/08 ,  G06N3/04 ,  G06K9/62

Abstract: One embodiment of the present invention sets forth a technique for performing spatial propagation. The technique includes generating a first directed acyclic graph (DAG) by connecting spatially adjacent points included in a set of unstructured points via directed edges along a first direction. The technique also includes applying a first set of neural network layers to one or more images associated with the set of unstructured points to generate (i) a set of features for the set of unstructured points and (ii) a set of pairwise affinities between the spatially adjacent points connected by the directed edges. The technique further includes generating a set of labels for the set of unstructured points by propagating the set of features across the first DAG based on the set of pairwise affinities.