公开(公告)号：US20230205219A1

公开(公告)日：2023-06-29

申请号：US18182060

申请日：2023-03-10

Applicant: NVIDIA Corporation

Inventor： Regan Blythe Towal ,  Maroof Mohammed Farooq ,  Vijay Chintalapudi ,  Carolina Parada ,  David Nister

IPC: G05D1/02 ,  G06T7/60 ,  G06N3/04 ,  G06V20/56 ,  G06V10/764 ,  G06V10/82

CPC classification number: G05D1/0221 ,  G06T7/60 ,  G06N3/04 ,  G06V20/56 ,  G06V10/764 ,  G06V10/82 ,  G06V20/588

Abstract: In various examples, a deep learning solution for path detection is implemented to generate a more abstract definition of a drivable path—without reliance on explicit lane-markings—by using a detection-based approach. Using approaches of the present disclosure, the identification of drivable paths may be possible in environments where conventional approaches are unreliable, or fail—such as where lane markings do not exist or are occluded. The deep learning solution may generate outputs that represent geometries for one or more drivable paths in an environment and confidence values corresponding to path types or classes that the geometries correspond. These outputs may be directly useable by an autonomous vehicle—such as an autonomous driving software stack—with minimal post-processing.

公开(公告)号：US20230037767A1

公开(公告)日：2023-02-09

申请号：US17395318

申请日：2021-08-05

Applicant: NVIDIA Corporation

Inventor： Fangkai Yang ,  David Nister ,  Yizhou Wang ,  Rotem Aviv ,  Julia Ng ,  Birgit Henke ,  Hon Leung Lee ,  Yunfei Shi

IPC: B60W60/00 ,  G08G1/0967 ,  B60W30/18

Abstract: In various examples, a yield scenario may be identified for a first vehicle. A wait element is received that encodes a first path for the first vehicle to traverse a yield area and a second path for a second vehicle to traverse the yield area. The first path is employed to determine a first trajectory in the yield area for the first vehicle based at least on a first location of the first vehicle at a time and the second path is employed to determine a second trajectory in the yield area for the second vehicle based at least on a second location of the second vehicle at the time. To operate the first vehicle in accordance with a wait state, it may be determined whether there is a conflict between the first trajectory and the second trajectory, where the wait state defines a yielding behavior for the first vehicle.

公开(公告)号：US11532168B2

公开(公告)日：2022-12-20

申请号：US16915346

申请日：2020-06-29

Applicant: NVIDIA Corporation

Inventor： Nikolai Smolyanskiy ,  Ryan Oldja ,  Ke Chen ,  Alexander Popov ,  Joachim Pehserl ,  Ibrahim Eden ,  Tilman Wekel ,  David Wehr ,  Ruchi Bhargava ,  David Nister

IPC: G06V20/58 ,  G01S17/931 ,  B60W60/00 ,  G01S17/89 ,  G05D1/00 ,  G06N3/04 ,  G06T19/00

Abstract: A deep neural network(s) (DNN) may be used to detect objects from sensor data of a three dimensional (3D) environment. For example, a multi-view perception DNN may include multiple constituent DNNs or stages chained together that sequentially process different views of the 3D environment. An example DNN may include a first stage that performs class segmentation in a first view (e.g., perspective view) and a second stage that performs class segmentation and/or regresses instance geometry in a second view (e.g., top-down). The DNN outputs may be processed to generate 2D and/or 3D bounding boxes and class labels for detected objects in the 3D environment. As such, the techniques described herein may be used to detect and classify animate objects and/or parts of an environment, and these detections and classifications may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.

公开(公告)号：US11508049B2

公开(公告)日：2022-11-22

申请号：US16570187

申请日：2019-09-13

Applicant: NVIDIA Corporation

Inventor： Hae-Jong Seo ,  Abhishek Bajpayee ,  David Nister ,  Minwoo Park ,  Neda Cvijetic

IPC: G06K9/00 ,  G06T7/00 ,  G06K9/62 ,  G06N3/04 ,  G06N3/08 ,  G06V10/98

Abstract: In various examples, a deep neural network (DNN) is trained for sensor blindness detection using a region and context-based approach. Using sensor data, the DNN may compute locations of blindness or compromised visibility regions as well as associated blindness classifications and/or blindness attributes associated therewith. In addition, the DNN may predict a usability of each instance of the sensor data for performing one or more operations—such as operations associated with semi-autonomous or autonomous driving. The combination of the outputs of the DNN may be used to filter out instances of the sensor data—or to filter out portions of instances of the sensor data determined to be compromised—that may lead to inaccurate or ineffective results for the one or more operations of the system.

公开(公告)号：US20220349725A1

公开(公告)日：2022-11-03

申请号：US17726429

申请日：2022-04-21

Applicant: NVIDIA Corporation

Inventor： Russell Chreptyk ,  Vaibhav Thukral ,  David Nister

IPC: G01C21/00 ,  B60W60/00

Abstract: In various examples, a high definition (HD) map is provided that includes a segmented data structure that allows for selective access to desired road segments and corresponding layers of map data. For example, the HD map may be segmented into a series of tiles that may correspond to a geographic region, and each of the tiles may include any number of road segments corresponding to portions of the geographic region. Each road segment may include a corresponding set of layers—which may include driving layers for use by the ego-machine and/or training layers for generating ground truth data—from the HD map that are associated with the road segment alone. As such, when traversing the environment, an ego-machine may determine one or more road segments within a tile corresponding to a current location, and may selectively download one or more layers for each of the one or more road segments.

公开(公告)号：US20220138568A1

公开(公告)日：2022-05-05

申请号：US17453055

申请日：2021-11-01

Applicant: NVIDIA Corporation

Inventor： Nikolai Smolyanskiy ,  Alexey Kamenev ,  Lirui Wang ,  David Nister ,  Ollin Boer Bohan ,  Ishwar Kulkarni ,  Fangkai Yang ,  Julia Ng ,  Alperen Degirmenci ,  Ruchi Bhargava ,  Rotem Aviv

IPC: G06N3/08 ,  G06N3/04

Abstract: In various examples, reinforcement learning is used to train at least one machine learning model (MLM) to control a vehicle by leveraging a deep neural network (DNN) trained on real-world data by using imitation learning to predict movements of one or more actors to define a world model. The DNN may be trained from real-world data to predict attributes of actors, such as locations and/or movements, from input attributes. The predictions may define states of the environment in a simulator, and one or more attributes of one or more actors input into the DNN may be modified or controlled by the simulator to simulate conditions that may otherwise be unfeasible. The MLM(s) may leverage predictions made by the DNN to predict one or more actions for the vehicle.

公开(公告)号：US20220135075A1

公开(公告)日：2022-05-05

申请号：US17497685

申请日：2021-10-08

Applicant: NVIDIA Corporation

Inventor： Julia Ng ,  Sachin Pullaikudi Veedu ,  David Nister ,  Hanne Buur ,  Hans Jonas Nilsson ,  Hon Leung Lee ,  Yunfei Shi ,  Charles Jerome Vorbach, JR.

IPC: B60W60/00 ,  B60W30/095 ,  G06F9/50

Abstract: In various examples, a safety decomposition architecture for autonomous machine applications is presented that uses two or more individual safety assessments to satisfy a higher safety integrity level (e.g., ASIL D). For example, a behavior planner may be used as a primary planning component, and a collision avoidance feature may be used as a diverse safety monitoring component—such that both may redundantly and independently prevent violation of safety goals. In addition, robustness of the system may be improved as single point and systematic failures may be avoided due to the requirement that two independent failures—e.g., of the behavior planner component and the collision avoidance component—occur simultaneously to cause a violation of the safety goals.

公开(公告)号：US11308338B2

公开(公告)日：2022-04-19

申请号：US16728595

申请日：2019-12-27

Applicant: NVIDIA Corporation

Inventor： Yilin Yang ,  Bala Siva Sashank Jujjavarapu ,  Pekka Janis ,  Zhaoting Ye ,  Sangmin Oh ,  Minwoo Park ,  Daniel Herrera Castro ,  Tommi Koivisto ,  David Nister

IPC: G06K9/00 ,  G06N3/08 ,  G06N3/04 ,  G06K9/52 ,  G06K9/20 ,  G06K9/62

Abstract: In various examples, a deep neural network (DNN) is trained to accurately predict, in deployment, distances to objects and obstacles using image data alone. The DNN may be trained with ground truth data that is generated and encoded using sensor data from any number of depth predicting sensors, such as, without limitation, RADAR sensors, LIDAR sensors, and/or SONAR sensors. Camera adaptation algorithms may be used in various embodiments to adapt the DNN for use with image data generated by cameras with varying parameters—such as varying fields of view. In some examples, a post-processing safety bounds operation may be executed on the predictions of the DNN to ensure that the predictions fall within a safety-permissible range.

公开(公告)号：US20210354729A1

公开(公告)日：2021-11-18

申请号：US16877127

申请日：2020-05-18

Applicant: NVIDIA Corporation

Inventor： Julia Ng ,  David Nister ,  Zhenyi Zhang ,  Yizhou Wang

IPC: B60W60/00 ,  B60W30/095

Abstract: In various examples, systems and methods are disclosed for weighting one or more optional paths based on obstacle avoidance or other safety considerations. In some embodiments, the obstacle avoidance considerations may be computed using a comparison of trajectories representative of safety procedures at present and future projected time steps of an ego-vehicle and other actors to ensure that each actor is capable of implementing their respective safety procedure while avoiding collisions at any point along the trajectory. This comparison may include filtering out a path(s) of an actor at a time step(s)—e.g., using a one-dimensional lookup—based on spatial relationships between the actor and the ego-vehicle at the time step(s). Where a particular path—or point along the path—does not satisfy a collision-free standard, the path may be penalized more negatively with respect to the obstacle avoidance considerations, or may be removed from consideration as a potential path.

公开(公告)号：US20210253128A1

公开(公告)日：2021-08-19

申请号：US17178464

申请日：2021-02-18

Applicant: NVIDIA Corporation

Inventor： David Nister ,  Yizhou Wang ,  Julia Ng ,  Rotem Aviv ,  Seungho Lee ,  Joshua John Bialkowski ,  Hon Leung Lee ,  Hermes Lanker ,  Raul Correal Tezanos ,  Zhenyi Zhang ,  Nikolai Smolyanskiy ,  Alexey Kamenev ,  Ollin Boer Bohan ,  Anton Vorontsov ,  Miguel Sainz Serra ,  Birgit Henke

IPC: B60W60/00 ,  G06N3/08 ,  G05D1/02 ,  B60W50/00

Abstract: Embodiments of the present disclosure relate to behavior planning for autonomous vehicles. The technology described herein selects a preferred trajectory for an autonomous vehicle based on an evaluation of multiple hypothetical trajectories by different components within a planning system. The various components provide an optimization score for each trajectory according to the priorities of the component and scores from multiple components may form a final optimization score. This scoring system allows the competing priorities (e.g., comfort, minimal travel time, fuel economy) of different components to be considered together. In examples, the trajectory with the best combined score may be selected for implementation. As such, an iterative approach that evaluates various factors may be used to identify an optimal or preferred trajectory for an autonomous vehicle when navigating an environment.