公开(公告)号：US20210201145A1

公开(公告)日：2021-07-01

申请号：US17116138

申请日：2020-12-09

Applicant: NVIDIA Corporation

Inventor： Trung Pham ,  Berta Rodriguez Hervas ,  Minwoo Park ,  David Nister ,  Neda Cvijetic

IPC: G06N3/08 ,  G06N3/04 ,  G06K9/00 ,  G06T7/33 ,  B60W30/18

Abstract: In various examples, a three-dimensional (3D) intersection structure may be predicted using a deep neural network (DNN) based on processing two-dimensional (2D) input data. To train the DNN to accurately predict 3D intersection structures from 2D inputs, the DNN may be trained using a first loss function that compares 3D outputs of the DNN—after conversion to 2D space—to 2D ground truth data and a second loss function that analyzes the 3D predictions of the DNN in view of one or more geometric constraints—e.g., geometric knowledge of intersections may be used to penalize predictions of the DNN that do not align with known intersection and/or road structure geometries. As such, live perception of an autonomous or semi-autonomous vehicle may be used by the DNN to detect 3D locations of intersection structures from 2D inputs.

公开(公告)号：US20210165418A1

公开(公告)日：2021-06-03

申请号：US17108965

申请日：2020-12-01

Applicant: NVIDIA Corporation

Inventor： Michael Grabner ,  Jeremy Furtek ,  David Nister

IPC: G05D1/02 ,  G06T7/246 ,  G06T7/285 ,  G06T7/35

Abstract: Systems and methods for performing visual odometry more rapidly. Pairs of representations from sensor data (such as images from one or more cameras) are selected, and features common to both representations of the pair are identified. Portions of bundle adjustment matrices that correspond to the pair are updated using the common features. These updates are maintained in register memory until all portions of the matrices that correspond to the pair are updated. By selecting only common features of one particular pair of representations, updated matrix values may be kept in registers. Accordingly, matrix updates for each common feature may be collectively saved with a single write of the registers to other memory. In this manner, fewer write operations are performed from register memory to other memory, thus reducing the time required to update bundle adjustment matrices and thus speeding the bundle adjustment process.

公开(公告)号：US20210156960A1

公开(公告)日：2021-05-27

申请号：US16836583

申请日：2020-03-31

Applicant: NVIDIA Corporation

Inventor： Alexander Popov ,  Nikolai Smolyanskiy ,  Ryan Oldja ,  Shane Murray ,  Tilman Wekel ,  David Nister ,  Joachim Pehserl ,  Ruchi Bhargava ,  Sangmin Oh

IPC: G01S7/41 ,  G06N3/08 ,  G06T7/73 ,  G06T7/246 ,  G01S13/931

Abstract: In various examples, a deep neural network(s) (e.g., a convolutional neural network) may be trained to detect moving and stationary obstacles from RADAR data of a three dimensional (3D) space, in both highway and urban scenarios. RADAR detections may be accumulated, ego-motion-compensated, orthographically projected, and fed into a neural network(s). The neural network(s) may include a common trunk with a feature extractor and several heads that predict different outputs such as a class confidence head that predicts a confidence map and an instance regression head that predicts object instance data for detected objects. The outputs may be decoded, filtered, and/or clustered to form bounding shapes identifying the location, size, and/or orientation of detected object instances. The detected object instances may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.

公开(公告)号：US20200026960A1

公开(公告)日：2020-01-23

申请号：US16514230

申请日：2019-07-17

Applicant: NVIDIA Corporation

Inventor： Minwoo Park ,  Xiaolin Lin ,  Hae-Jong Seo ,  David Nister ,  Neda Cvijetic

IPC: G06K9/62 ,  G06N3/04 ,  G05D1/00 ,  G06K9/00

Abstract: In various examples, systems and methods are disclosed that preserve rich spatial information from an input resolution of a machine learning model to regress on lines in an input image. The machine learning model may be trained to predict, in deployment, distances for each pixel of the input image at an input resolution to a line pixel determined to correspond to a line in the input image. The machine learning model may further be trained to predict angles and label classes of the line. An embedding algorithm may be used to train the machine learning model to predict clusters of line pixels that each correspond to a respective line in the input image. In deployment, the predictions of the machine learning model may be used as an aid for understanding the surrounding environment—e.g., for updating a world model—in a variety of autonomous machine applications.

公开(公告)号：US20190384304A1

公开(公告)日：2019-12-19

申请号：US16433994

申请日：2019-06-06

Applicant: NVIDIA Corporation

Inventor： Regan Blythe Towal ,  Maroof Mohammed Farooq ,  Vijay Chintalapudi ,  Carolina Parada ,  David Nister

IPC: G05D1/02 ,  G06K9/00 ,  G06N3/04 ,  G06T7/60

Abstract: In various examples, a deep learning solution for path detection is implemented to generate a more abstract definition of a drivable path without reliance on explicit lane-markings—by using a detection-based approach. Using approaches of the present disclosure, the identification of drivable paths may be possible in environments where conventional approaches are unreliable, or fail—such as where lane markings do not exist or are occluded. The deep learning solution may generate outputs that represent geometries for one or more drivable paths in an environment and confidence values corresponding to path types or classes that the geometries correspond. These outputs may be directly useable by an autonomous vehicle—such as an autonomous driving software stack—with minimal post-processing.

公开(公告)号：US12299892B2

公开(公告)日：2025-05-13

申请号：US18391276

申请日：2023-12-20

Applicant: NVIDIA Corporation

Inventor： Trung Pham ,  Berta Rodriguez Hervas ,  Minwoo Park ,  David Nister ,  Neda Cvijetic

IPC: G06T7/11 ,  G05B13/02 ,  G06F18/21 ,  G06F18/24 ,  G06N3/04 ,  G06N3/08 ,  G06T3/4046 ,  G06T5/70 ,  G06T11/20 ,  G06V10/26 ,  G06V10/34 ,  G06V10/44 ,  G06V10/82 ,  G06V20/56 ,  G06V30/19 ,  G06V30/262

Abstract: In various examples, live perception from sensors of a vehicle may be leveraged to detect and classify intersection contention areas in an environment of a vehicle in real-time or near real-time. For example, a deep neural network (DNN) may be trained to compute outputs—such as signed distance functions—that may correspond to locations of boundaries delineating intersection contention areas. The signed distance functions may be decoded and/or post-processed to determine instance segmentation masks representing locations and classifications of intersection areas or regions. The locations of the intersections areas or regions may be generated in image-space and converted to world-space coordinates to aid an autonomous or semi-autonomous vehicle in navigating intersections according to rules of the road, traffic priority considerations, and/or the like.

公开(公告)号：US20250138530A1

公开(公告)日：2025-05-01

申请号：US19005734

申请日：2024-12-30

Applicant: NVIDIA Corporation

Inventor： Minwoo Park ,  Xiaolin Lin ,  Hae-Jong Seo ,  David Nister ,  Neda Cvijetic

IPC: G05D1/00 ,  G05D1/228 ,  G06F18/214 ,  G06F18/23 ,  G06F18/2411 ,  G06N3/04 ,  G06V10/44 ,  G06V10/48 ,  G06V10/75 ,  G06V10/764 ,  G06V10/776 ,  G06V10/82 ,  G06V10/94 ,  G06V20/56

Abstract: In various examples, systems and methods are disclosed that preserve rich spatial information from an input resolution of a machine learning model to regress on lines in an input image. The machine learning model may be trained to predict, in deployment, distances for each pixel of the input image at an input resolution to a line pixel determined to correspond to a line in the input image. The machine learning model may further be trained to predict angles and label classes of the line. An embedding algorithm may be used to train the machine learning model to predict clusters of line pixels that each correspond to a respective line in the input image. In deployment, the predictions of the machine learning model may be used as an aid for understanding the surrounding environment—e.g., for updating a world model—in a variety of autonomous machine applications.

公开(公告)号：US12269488B2

公开(公告)日：2025-04-08

申请号：US17726407

申请日：2022-04-21

Applicant: NVIDIA Corporation

Inventor： David Nister ,  Cheng-Chieh Yang ,  Yue Wu

IPC: B60W50/02 ,  B60W60/00

Abstract: In various examples, an end-to-end perception evaluation system for autonomous and semi-autonomous machine applications may be implemented to evaluate how the accuracy or precision of outputs of machine learning models—such as deep neural networks (DNNs)—impact downstream performance of the machine when relied upon. For example, decisions computed by the system using ground truth output types may be compared to decisions computed by the system using the perception outputs. As a result, discrepancies in downstream decision making of the system between the ground truth information and the perception information may be evaluated to either aid in updating or retraining of the machine learning model or aid in generating more accurate or precise ground truth information.

公开(公告)号：US20250100581A1

公开(公告)日：2025-03-27

申请号：US18974191

申请日：2024-12-09

Applicant: NVIDIA Corporation

Inventor： Birgit Henke ,  David Nister ,  Julia Ng

IPC: B60W60/00

Abstract: A trajectory for an autonomous machine may be evaluated for safety based at least on determining whether the autonomous machine would be capable of occupying points of the trajectory in space-time while still being able to avoid a potential future collision with one or more objects in the environment through use of one or more safety procedures. To do so, a point of the trajectory may be evaluated for conflict based at least on a comparison between points in space-time that correspond to the autonomous machine executing the safety procedure(s) from the point and arrival times of the one or more objects to corresponding position(s) in the environment. A trajectory may be sampled and evaluated for conflicts at various points throughout the trajectory. Based on results of one or more evaluations, the trajectory may be scored, eliminated from consideration, or otherwise considered for control of the autonomous machine.

公开(公告)号：US20250029264A1

公开(公告)日：2025-01-23

申请号：US18905939

申请日：2024-10-03

Applicant: NVIDIA Corporation

Inventor： David Nister ,  Soohwan Kim ,  Yue Wu ,  Minwoo Park ,  Cheng-Chieh Yang

IPC: G06T7/215 ,  G06T7/60 ,  G06V10/422

Abstract: In various examples, an ego-machine may analyze sensor data to identify and track features in the sensor data using. Geometry of the tracked features may be used to analyze motion flow to determine whether the motion flow violates one or more geometrical constraints. As such, tracked features may be identified as dynamic features when the motion flow corresponding to the tracked features violates the one or more static constraints for static features. Tracked features that are determined to be dynamic features may be clustered together according to their location and feature track. Once features have been clustered together, the system may calculate a detection bounding shape for the clustered features. The bounding shape information may then be used by the ego-machine for path planning, control decisions, obstacle avoidance, and/or other operations.