公开(公告)号：US20210272304A1

公开(公告)日：2021-09-02

申请号：US16728598

申请日：2019-12-27

Applicant: NVIDIA Corporation

Inventor： Yilin Yang ,  Bala Siva Sashank Jujjavarapu ,  Pekka Janis ,  Zhaoting Ye ,  Sangmin Oh ,  Minwoo Park ,  Daniel Herrera Castro ,  Tommi Koivisto ,  David Nister

IPC: G06T7/536 ,  G06K9/00

Abstract: In various examples, a deep neural network (DNN) is trained to accurately predict, in deployment, distances to objects and obstacles using image data alone. The DNN may be trained with ground truth data that is generated and encoded using sensor data from any number of depth predicting sensors, such as, without limitation, RADAR sensors, LIDAR sensors, and/or SONAR sensors. Camera adaptation algorithms may be used in various embodiments to adapt the DNN for use with image data generated by cameras with varying parameters—such as varying fields of view. In some examples, a post-processing safety bounds operation may be executed on the predictions of the DNN to ensure that the predictions fall within a safety-permissible range.

公开(公告)号：US11079764B2

公开(公告)日：2021-08-03

申请号：US16265780

申请日：2019-02-01

Applicant: NVIDIA Corporation

Inventor： David Nister ,  Hon-Leung Lee ,  Julia Ng ,  Yizhou Wang

IPC: G05D1/02 ,  B60W30/09 ,  G05D1/08 ,  B60W30/095

Abstract: In various examples, a current claimed set of points representative of a volume in an environment occupied by a vehicle at a time may be determined. A vehicle-occupied trajectory and at least one object-occupied trajectory may be generated at the time. An intersection between the vehicle-occupied trajectory and an object-occupied trajectory may be determined based at least in part on comparing the vehicle-occupied trajectory to the object-occupied trajectory. Based on the intersection, the vehicle may then execute the first safety procedure or an alternative procedure that, when implemented by the vehicle when the object implements the second safety procedure, is determined to have a lesser likelihood of incurring a collision between the vehicle and the object than the first safety procedure.

公开(公告)号：US20210156963A1

公开(公告)日：2021-05-27

申请号：US16836618

申请日：2020-03-31

Applicant: NVIDIA Corporation

Inventor： Alexander Popov ,  Nikolai Smolyanskiy ,  Ryan Oldja ,  Shane Murray ,  Tilman Wekel ,  David Nister ,  Joachim Pehserl ,  Ruchi Bhargava ,  Sangmin Oh

IPC: G01S7/41 ,  G01S13/86 ,  G01S13/89 ,  G06N3/08 ,  G06N3/04

Abstract: In various examples, a deep neural network(s) (e.g., a convolutional neural network) may be trained to detect moving and stationary obstacles from RADAR data of a three dimensional (3D) space. In some embodiments, ground truth training data for the neural network(s) may be generated from LIDAR data. More specifically, a scene may be observed with RADAR and LIDAR sensors to collect RADAR data and LIDAR data for a particular time slice. The RADAR data may be used for input training data, and the LIDAR data associated with the same or closest time slice as the RADAR data may be annotated with ground truth labels identifying objects to be detected. The LIDAR labels may be propagated to the RADAR data, and LIDAR labels containing less than some threshold number of RADAR detections may be omitted. The (remaining) LIDAR labels may be used to generate ground truth data.

公开(公告)号：US20210150230A1

公开(公告)日：2021-05-20

申请号：US16915346

申请日：2020-06-29

Applicant: NVIDIA Corporation

Inventor： Nikolai Smolyanskiy ,  Ryan Oldja ,  Ke Chen ,  Alexander Popov ,  Joachim Pehserl ,  Ibrahim Eden ,  Tilman Wekel ,  David Wehr ,  Ruchi Bhargava ,  David Nister

IPC: G06K9/00 ,  B60W60/00 ,  G05D1/00 ,  G06N3/04 ,  G06T19/00 ,  G01S17/89 ,  G01S17/931

Abstract: A deep neural network(s) (DNN) may be used to detect objects from sensor data of a three dimensional (3D) environment. For example, a multi-view perception DNN may include multiple constituent DNNs or stages chained together that sequentially process different views of the 3D environment. An example DNN may include a first stage that performs class segmentation in a first view (e.g., perspective view) and a second stage that performs class segmentation and/or regresses instance geometry in a second view (e.g., top-down). The DNN outputs may be processed to generate 2D and/or 3D bounding boxes and class labels for detected objects in the 3D environment. As such, the techniques described herein may be used to detect and classify animate objects and/or parts of an environment, and these detections and classifications may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.

公开(公告)号：US20210063198A1

公开(公告)日：2021-03-04

申请号：US17008074

申请日：2020-08-31

Applicant: NVIDIA Corporation

Inventor： David Nister ,  Ruchi Bhargava ,  Vaibhav Thukral ,  Michael Grabner ,  Ibrahim Eden ,  Jeffrey Liu

IPC: G01C21/00 ,  G01C21/16 ,  G06N3/02

Abstract: An end-to-end system for data generation, map creation using the generated data, and localization to the created map is disclosed. Mapstreams—or streams of sensor data, perception outputs from deep neural networks (DNNs), and/or relative trajectory data—corresponding to any number of drives by any number of vehicles may be generated and uploaded to the cloud. The mapstreams may be used to generate map data—and ultimately a fused high definition (HD) map—that represents data generated over a plurality of drives. When localizing to the fused HD map, individual localization results may be generated based on comparisons of real-time data from a sensor modality to map data corresponding to the same sensor modality. This process may be repeated for any number of sensor modalities and the results may be fused together to determine a final fused localization result.

公开(公告)号：US20200339109A1

公开(公告)日：2020-10-29

申请号：US16860824

申请日：2020-04-28

Applicant: NVIDIA Corporation

Inventor： Jesse Hong ,  Urs Muller ,  Bernhard Firner ,  Zongyi Yang ,  Joyjit Daw ,  David Nister ,  Roberto Giuseppe Luca Valenti ,  Rotem Aviv

IPC: B60W30/08 ,  B60W30/12 ,  B60W30/14 ,  B60W60/00

Abstract: In various examples, sensor data recorded in the real-world may be leveraged to generate transformed, additional, sensor data to test one or more functions of a vehicle—such as a function of an AEB, CMW, LDW, ALC, or ACC system. Sensor data recorded by the sensors may be augmented, transformed, or otherwise updated to represent sensor data corresponding to state information defined by a simulation test profile for testing the vehicle function(s). Once a set of test data has been generated, the test data may be processed by a system of the vehicle to determine the efficacy of the system with respect to any number of test criteria. As a result, a test set including additional or alternative instances of sensor data may be generated from real-world recorded sensor data to test a vehicle in a variety of test scenarios—including those that may be too dangerous to test in the real-world.

公开(公告)号：US20200294310A1

公开(公告)日：2020-09-17

申请号：US16820164

申请日：2020-03-16

Applicant: NVIDIA CORPORATION

Inventor： Dongwoo Lee ,  Junghyun Kwon ,  Sangmin Oh ,  Wenchao Zheng ,  Hae-Jong Seo ,  David Nister ,  Berta Rodriguez Hervas

IPC: G06T17/30 ,  G06T7/40

Abstract: A neural network may be used to determine corner points of a skewed polygon (e.g., as displacement values to anchor box corner points) that accurately delineate a region in an image that defines a parking space. Further, the neural network may output confidence values predicting likelihoods that corner points of an anchor box correspond to an entrance to the parking spot. The confidence values may be used to select a subset of the corner points of the anchor box and/or skewed polygon in order to define the entrance to the parking spot. A minimum aggregate distance between corner points of a skewed polygon predicted using the CNN(s) and ground truth corner points of a parking spot may be used simplify a determination as to whether an anchor box should be used as a positive sample for training.

公开(公告)号：US10776983B2

公开(公告)日：2020-09-15

申请号：US16051263

申请日：2018-07-31

Applicant: Nvidia Corporation

Inventor： Ishwar Kulkarni ,  Ibrahim Eden ,  Michael Kroepfl ,  David Nister

IPC: G01S17/58 ,  G01S17/89 ,  G01S17/931 ,  G01K9/00 ,  G06T11/00 ,  G06T15/04 ,  G06T7/20 ,  G06T7/30 ,  G06T7/521 ,  G06K9/00

Abstract: Various types of systems or technologies can be used to collect data in a 3D space. For example, LiDAR (light detection and ranging) and RADAR (radio detection and ranging) systems are commonly used to generate point cloud data for 3D space around vehicles, for such functions as localization, mapping, and tracking. This disclosure provides improvements for processing the point cloud data that has been collected. The processing improvements include analyzing point cloud data using trajectory equations, depth maps, and texture maps. The processing improvements also include representing the point cloud data by a two dimensional depth map or a texture map and using the depth map or texture map to provide object motion, obstacle detection, freespace detection, and landmark detection for an area surrounding a vehicle.

公开(公告)号：US12164059B2

公开(公告)日：2024-12-10

申请号：US17377064

申请日：2021-07-15

Applicant: NVIDIA Corporation

Inventor： Nikolai Smolyanskiy ,  Ryan Oldja ,  Ke Chen ,  Alexander Popov ,  Joachim Pehserl ,  Ibrahim Eden ,  Tilman Wekel ,  David Wehr ,  Ruchi Bhargava ,  David Nister

IPC: G01S7/48 ,  B60W60/00 ,  G01S17/89 ,  G01S17/931 ,  G05D1/00 ,  G06N3/045 ,  G06T19/00 ,  G06V10/10 ,  G06V10/25 ,  G06V10/26 ,  G06V10/44 ,  G06V10/764 ,  G06V10/774 ,  G06V10/80 ,  G06V10/82 ,  G06V20/56 ,  G06V20/58

Abstract: A deep neural network(s) (DNN) may be used to detect objects from sensor data of a three dimensional (3D) environment. For example, a multi-view perception DNN may include multiple constituent DNNs or stages chained together that sequentially process different views of the 3D environment. An example DNN may include a first stage that performs class segmentation in a first view (e.g., perspective view) and a second stage that performs class segmentation and/or regresses instance geometry in a second view (e.g., top-down). The DNN outputs may be processed to generate 2D and/or 3D bounding boxes and class labels for detected objects in the 3D environment. As such, the techniques described herein may be used to detect and classify animate objects and/or parts of an environment, and these detections and classifications may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.

公开(公告)号：US20240400097A1

公开(公告)日：2024-12-05

申请号：US18674551

申请日：2024-05-24

Applicant: NVIDIA Corporation

Inventor： David Nister

IPC: B60W60/00 ,  B60W50/00

Abstract: Costs associated with configurations corresponding to a maneuver type(s) may be stored in a transition state(s) volume. The same memory volume may be used for storing cost values that correspond different maneuver types and different vertices in a graph of a configuration space. In at least one embodiment, to share a memory volume between maneuver types, the system may determine a cost for a machine to reach a configuration of a configuration space using various different maneuver types. The system may then evaluate one or more of the costs to determine which of the costs to store at one or more memory location(s) corresponding to the configuration (e.g., a point in a memory volume). Cost values for the memory volume may be efficiently determined using kernel-style processing.