公开(公告)号：US20210042575A1

公开(公告)日：2021-02-11

申请号：US16874003

申请日：2020-05-14

Applicant: NVIDIA Corporation

Inventor： Bernhard Firner

IPC: G06K9/62 ,  G06N3/08 ,  G06N3/04 ,  G05D1/00 ,  G05D1/02

Abstract: A neural network is trained to focus on a domain of interest. For example, in a pre-training phase, the neural network in trained using synthetic training data, which is configured to omit or limit content less relevant to the domain of interest, by updating parameters of the neural network to improve the accuracy of predictions. In a subsequent training phase, the pre-trained neural network is trained using real-world training data by updating only a first subset of the parameters associated with feature extraction, while a second subset of the parameters more associated with policies remains fixed.

公开(公告)号：US11841458B2

公开(公告)日：2023-12-12

申请号：US17947743

申请日：2022-09-19

Applicant: NVIDIA Corporation

Inventor： Bernhard Firner

IPC: G01S7/41 ,  G06N3/08 ,  G06N3/04 ,  G05D1/00 ,  G05D1/02 ,  G06F18/214 ,  G06F18/24 ,  G06F18/213 ,  G06V10/764 ,  G06V10/774 ,  G06V10/82 ,  G06V20/17 ,  G06V20/10 ,  G06V20/56

CPC classification number: G01S7/417 ,  G05D1/0088 ,  G05D1/0221 ,  G05D1/0251 ,  G06F18/213 ,  G06F18/214 ,  G06F18/24 ,  G06N3/04 ,  G06N3/08 ,  G06V10/764 ,  G06V10/774 ,  G06V10/82 ,  G06V20/17 ,  G06V20/176 ,  G06V20/56 ,  G05D2201/0213

Abstract: A neural network is trained to focus on a domain of interest. For example, in a pre-training phase, the neural network in trained using synthetic training data, which is configured to omit or limit content less relevant to the domain of interest, by updating parameters of the neural network to improve the accuracy of predictions. In a subsequent training phase, the pre-trained neural network is trained using real-world training data by updating only a first subset of the parameters associated with feature extraction, while a second subset of the parameters more associated with policies remains fixed.

公开(公告)号：US20220092317A1

公开(公告)日：2022-03-24

申请号：US17448247

申请日：2021-09-21

Applicant: NVIDIA Corporation

Inventor： Zongyi Yang ,  Mariusz Bojarski ,  Bernhard Firner ,  Urs Muller

IPC: G06K9/00 ,  G05B13/02 ,  G06K9/32 ,  G06T5/00 ,  G06K9/62 ,  G06T3/60 ,  G06T7/11

Abstract: In various examples, sensor data used to train an MLM and/or used by the MLM during deployment, may be captured by sensors having different perspectives (e.g., fields of view). The sensor data may be transformed—to generate transformed sensor data—such as by altering or removing lens distortions, shifting, and/or rotating images corresponding to the sensor data to a field of view of a different physical or virtual sensor. As such, the MLM may be trained and/or deployed using sensor data captured from a same or similar field of view. As a result, the MLM may be trained and/or deployed—across any number of different vehicles with cameras and/or other sensors having different perspectives—using sensor data that is of the same perspective as the reference or ideal sensor.

公开(公告)号：US20190384303A1

公开(公告)日：2019-12-19

申请号：US16409056

申请日：2019-05-10

Applicant: NVIDIA Corporation

Inventor： Urs Muller ,  Mariusz Bojarski ,  Chenyi Chen ,  Bernhard Firner

IPC: G05D1/02 ,  G06N3/08 ,  G06N20/00

Abstract: In various examples, a machine learning model—such as a deep neural network (DNN)—may be trained to use image data and/or other sensor data as inputs to generate two-dimensional or three-dimensional trajectory points in world space, a vehicle orientation, and/or a vehicle state. For example, sensor data that represents orientation, steering information, and/or speed of a vehicle may be collected and used to automatically generate a trajectory for use as ground truth data for training the DNN. Once deployed, the trajectory points, the vehicle orientation, and/or the vehicle state may be used by a control component (e.g., a vehicle controller) for controlling the vehicle through a physical environment. For example, the control component may use these outputs of the DNN to determine a control profile (e.g., steering, decelerating, and/or accelerating) specific to the vehicle for controlling the vehicle through the physical environment.

公开(公告)号：US20250069385A1

公开(公告)日：2025-02-27

申请号：US18945136

申请日：2024-11-12

Applicant: NVIDIA Corporation

Inventor： Haiguang Wen ,  Bernhard Firner ,  Mariusz Bojarski ,  Zongyi Yang ,  Urs Muller

IPC: G06V10/82 ,  G06N3/08 ,  G06T7/70 ,  G06T9/00 ,  G06V10/25 ,  G06V10/50 ,  G06V10/52 ,  G06V20/56

Abstract: In examples, image data representative of an image of a field of view of at least one sensor may be received. Source areas may be defined that correspond to a region of the image. Areas and/or dimensions of at least some of the source areas may decrease along at least one direction relative to a perspective of the at least one sensor. A downsampled version of the region (e.g., a downsampled image or feature map of a neural network) may be generated from the source areas based at least in part on mapping the source areas to cells of the downsampled version of the region. Resolutions of the region that are captured by the cells may correspond to the areas of the source areas, such that certain portions of the region (e.g., portions at a far distance from the sensor) retain higher resolution than others.

公开(公告)号：US20240183752A1

公开(公告)日：2024-06-06

申请号：US18442753

申请日：2024-02-15

Applicant: NVIDIA Corporation

Inventor： Jesse Hong ,  Urs Muller ,  Bernhard Firner ,  Zongyi Yang ,  Joyjit Daw ,  David Nister ,  Roberto Giuseppe Luca Valenti ,  Rotem Aviv

IPC: G01M17/007 ,  B60W30/08 ,  B60W30/12 ,  B60W30/14 ,  B60W50/00 ,  B60W50/04 ,  B60W60/00 ,  G06F11/36 ,  G06V10/774 ,  G06V20/56 ,  G07C5/08

CPC classification number: G01M17/007 ,  B60W30/08 ,  B60W30/12 ,  B60W30/143 ,  B60W50/04 ,  B60W50/045 ,  B60W60/0011 ,  G06V10/774 ,  G06V20/56 ,  G07C5/08 ,  B60W2050/0028 ,  G06F11/3684 ,  G06F11/3696

Abstract: In various examples, sensor data recorded in the real-world may be leveraged to generate transformed, additional, sensor data to test one or more functions of a vehicle—such as a function of an AEB, CMW, LDW, ALC, or ACC system. Sensor data recorded by the sensors may be augmented, transformed, or otherwise updated to represent sensor data corresponding to state information defined by a simulation test profile for testing the vehicle function(s). Once a set of test data has been generated, the test data may be processed by a system of the vehicle to determine the efficacy of the system with respect to any number of test criteria. As a result, a test set including additional or alternative instances of sensor data may be generated from real-world recorded sensor data to test a vehicle in a variety of test scenarios.

公开(公告)号：US20240127062A1

公开(公告)日：2024-04-18

申请号：US18533860

申请日：2023-12-08

Applicant: NVIDIA Corporation

Inventor： Urs Muller ,  Mariusz Bojarski ,  Chenyi Chen ,  Bernhard Firner

IPC: G06N3/08 ,  G06N20/00 ,  G06V10/774 ,  G06V20/56

CPC classification number: G06N3/08 ,  G06N20/00 ,  G06V10/774 ,  G06V20/56

Abstract: In various examples, a machine learning model—such as a deep neural network (DNN)—may be trained to use image data and/or other sensor data as inputs to generate two-dimensional or three-dimensional trajectory points in world space, a vehicle orientation, and/or a vehicle state. For example, sensor data that represents orientation, steering information, and/or speed of a vehicle may be collected and used to automatically generate a trajectory for use as ground truth data for training the DNN. Once deployed, the trajectory points, the vehicle orientation, and/or the vehicle state may be used by a control component (e.g., a vehicle controller) for controlling the vehicle through a physical environment. For example, the control component may use these outputs of the DNN to determine a control profile (e.g., steering, decelerating, and/or accelerating) specific to the vehicle for controlling the vehicle through the physical environment.

公开(公告)号：US20230110713A1

公开(公告)日：2023-04-13

申请号：US17497479

申请日：2021-10-08

Applicant: NVIDIA Corporation

Inventor： Alperen Degirmenci ,  Won Hong ,  Mariusz Bojarski ,  Jesper Eduard van Engelen ,  Bernhard Firner ,  Zongyi Yang ,  Urs Muller

IPC: G05D1/02 ,  G06N3/00 ,  B60W60/00

Abstract: In various examples, a plurality of poses corresponding to one or more configuration parameters within an environment—such as a location of a machine within an environment, an orientation of a machine within an environment, a sensor angle pose of a machine, or a sensor location of a machine—may be used to generate training data and corresponding ground truth data for training a machine learning model—such as a deep neural network (DNN). As a result, the machine learning model, once deployed, may more accurately compute one or more outputs—such as outputs representative of lane boundaries, trajectories for an autonomous machine, etc.—agnostic to machine and/or sensor poses of the machine within which the machine learning model is deployed.

公开(公告)号：US12183063B2

公开(公告)日：2024-12-31

申请号：US16917289

申请日：2020-06-30

Applicant: NVIDIA Corporation

Inventor： Haiguang Wen ,  Bernhard Firner ,  Mariusz Bojarski ,  Zongyi Yang ,  Urs Muller

IPC: G06K9/00 ,  G06N3/08 ,  G06T7/70 ,  G06T9/00 ,  G06V10/25 ,  G06V10/50 ,  G06V10/52 ,  G06V10/82 ,  G06V20/56

Abstract: In examples, image data representative of an image of a field of view of at least one sensor may be received. Source areas may be defined that correspond to a region of the image. Areas and/or dimensions of at least some of the source areas may decrease along at least one direction relative to a perspective of the at least one sensor. A downsampled version of the region (e.g., a downsampled image or feature map of a neural network) may be generated from the source areas based at least in part on mapping the source areas to cells of the downsampled version of the region. Resolutions of the region that are captured by the cells may correspond to the areas of the source areas, such that certain portions of the region (e.g., portions at a far distance from the sensor) retain higher resolution than others.

公开(公告)号：US11927502B2

公开(公告)日：2024-03-12

申请号：US16860824

申请日：2020-04-28

Applicant: NVIDIA Corporation

Inventor： Jesse Hong ,  Urs Muller ,  Bernhard Firner ,  Zongyi Yang ,  Joyjit Daw ,  David Nister ,  Roberto Giuseppe Luca Valenti ,  Rotem Aviv

IPC: G01M17/007 ,  B60W30/08 ,  B60W30/12 ,  B60W30/14 ,  B60W50/00 ,  B60W50/04 ,  B60W60/00 ,  G06V10/774 ,  G06V20/56 ,  G07C5/08 ,  G06F11/36

CPC classification number: G01M17/007 ,  B60W30/08 ,  B60W30/12 ,  B60W30/143 ,  B60W50/04 ,  B60W50/045 ,  B60W60/0011 ,  G06V10/774 ,  G06V20/56 ,  G07C5/08 ,  B60W2050/0028 ,  G06F11/3684 ,  G06F11/3696

Abstract: In various examples, sensor data recorded in the real-world may be leveraged to generate transformed, additional, sensor data to test one or more functions of a vehicle—such as a function of an AEB, CMW, LDW, ALC, or ACC system. Sensor data recorded by the sensors may be augmented, transformed, or otherwise updated to represent sensor data corresponding to state information defined by a simulation test profile for testing the vehicle function(s). Once a set of test data has been generated, the test data may be processed by a system of the vehicle to determine the efficacy of the system with respect to any number of test criteria. As a result, a test set including additional or alternative instances of sensor data may be generated from real-world recorded sensor data to test a vehicle in a variety of test scenarios—including those that may be too dangerous to test in the real-world.