公开(公告)号：US20240281988A1

公开(公告)日：2024-08-22

申请号：US18171016

申请日：2023-02-17

Applicant: NVIDIA Corporation

Inventor： Joshua Edward ABBOTT ,  Amir AKBARZADEH ,  Joachim PEHSERL ,  Samuel Ogden ,  David WEHR ,  Ke CHEN

IPC: G06T7/50 ,  G01S17/89

CPC classification number: G06T7/50 ,  G01S17/89 ,  G06T2207/10028 ,  G06T2207/20084 ,  G06T2207/30256

Abstract: In various examples, perception of landmark shapes may be used for localization in autonomous systems and applications. In some embodiments, a deep neural network (DNN) is used to generate (e.g., per-point) classifications of measured 3D points (e.g., classified LiDAR points), and a representation of the shape of one or more detected landmarks is regressed from the classifications. For each of one or more classes, the classification data may be thresholded to generate a binary mask and/or dilated to generate a densified representation, and the resulting (e.g., dilated, binary) mask may be clustered into connected components that are iteratively: fitted a shape (e.g., a polynomial or Bezier spline for lane lines, a circle for top-down representations of poles or traffic lights), weighted, and merged. As such, the resulting connected components and their fitted shapes may be used to represent detected landmarks and used for localization, navigation, and/or other uses.

公开(公告)号：US20240280372A1

公开(公告)日：2024-08-22

申请号：US18171004

申请日：2023-02-17

Applicant: NVIDIA Corporation

Inventor： Joshua Edward ABBOTT ,  Amir AKBARZADEH ,  Joachim PEHSERL ,  Samuel OGDEN ,  David WEHR ,  Ke CHEN

IPC: G01C21/36 ,  B60W60/00 ,  G06V10/82

CPC classification number: G01C21/3644 ,  B60W60/001 ,  G06V10/82 ,  B60W2420/403

Abstract: In various examples, one or more DNNs may be used to detect landmarks (e.g., lane lines) and regress a representation of their shape. A DNN may be used to jointly generate classifications of measured 3D points using one output head (e.g., a classification head) and regress a representation of one or more fitted shapes (e.g., polylines, circles) using a second output head (e.g., a regression head). In some embodiments, multiple DNNs (e.g., a chain of multiple DNNs or multiple stages of a DNN) are used to sequentially generate classifications of measured 3D points and a regressed representation of the shape of one or more detected landmarks. As such, classified landmarks and corresponding fitted shapes may be decoded and used for localization, navigation, and/or other uses.

公开(公告)号：US20240096102A1

公开(公告)日：2024-03-21

申请号：US18366298

申请日：2023-08-07

Applicant: NVIDIA Corporation

Inventor： Alexander POPOV ,  David NISTER ,  Nikolai SMOLYANSKIY ,  PATRIK GEBHARDT ,  Ke CHEN ,  Ryan OLDJA ,  Hee Seok LEE ,  Shane MURRAY ,  Ruchi BHARGAVA ,  Tilman WEKEL ,  Sangmin OH

IPC: G06V20/56 ,  G01S13/89 ,  G01S17/89 ,  G06V10/774

CPC classification number: G06V20/56 ,  G01S13/89 ,  G01S17/89 ,  G06V10/774

Abstract: Systems and methods are disclosed that relate to freespace detection using machine learning models. First data that may include object labels may be obtained from a first sensor and freespace may be identified using the first data and the object labels. The first data may be annotated to include freespace labels that correspond to freespace within an operational environment. Freespace annotated data may be generated by combining the one or more freespace labels with second data obtained from a second sensor, with the freespace annotated data corresponding to a viewable area in the operational environment. The viewable area may be determined by tracing one or more rays from the second sensor within the field of view of the second sensor relative to the first data. The freespace annotated data may be input into a machine learning model to train the machine learning model to detect freespace using the second data.

公开(公告)号：US20250014186A1

公开(公告)日：2025-01-09

申请号：US18397921

申请日：2023-12-27

Applicant: NVIDIA Corporation

Inventor： Ke CHEN ,  Nikolai SMOLYANSKIY ,  Alexey KAMENEV ,  Ryan OLDJA ,  Tilman WEKEL ,  David NISTER ,  Joachim PEHSERL ,  Ibrahim EDEN ,  Sangmin OH ,  Ruchi BHARGAVA

IPC: G06T7/11 ,  G05D1/81 ,  G06F18/22 ,  G06F18/23 ,  G06T5/50 ,  G06T7/10 ,  G06V10/44 ,  G06V10/82 ,  G06V20/56 ,  G06V20/58

Abstract: A deep neural network(s) (DNN) may be used to perform panoptic segmentation by performing pixel-level class and instance segmentation of a scene using a single pass of the DNN. Generally, one or more images and/or other sensor data may be stitched together, stacked, and/or combined, and fed into a DNN that includes a common trunk and several heads that predict different outputs. The DNN may include a class confidence head that predicts a confidence map representing pixels that belong to particular classes, an instance regression head that predicts object instance data for detected objects, an instance clustering head that predicts a confidence map of pixels that belong to particular instances, and/or a depth head that predicts range values. These outputs may be decoded to identify bounding shapes, class labels, instance labels, and/or range values for detected objects, and used to enable safe path planning and control of an autonomous vehicle.

公开(公告)号：US20240386586A1

公开(公告)日：2024-11-21

申请号：US18320265

申请日：2023-05-19

Applicant: NVIDIA Corporation

Inventor： Alperen DEGIRMENCI ,  Jiwoong CHOI ,  Zhiding YU ,  Ke CHEN ,  Shubhranshu SINGH ,  Yashar ASGARIEH ,  Subhashree RADHAKRISHNAN ,  James SKINNER ,  Jose Manuel ALVAREZ LOPEZ

IPC: G06T7/50 ,  G06T3/00

Abstract: In various examples, systems and methods are disclosed relating to using neural networks for object detection or instance/semantic segmentation for, without limitation, autonomous or semi-autonomous systems and applications. In some implementations, one or more neural networks receive an image (or other sensor data representation) and a bounding shape corresponding to at least a portion of an object in the image. The bounding shape can include or be labeled with an identifier, class, and/or category of the object. The neural network can determine a mask for the object based at least on processing the image and the bounding shape. The mask can be used for various applications, such as annotating masks for vehicle or machine perception and navigation processes.