公开(公告)号：US20230298361A1

公开(公告)日：2023-09-21

申请号：US17696083

申请日：2022-03-16

Applicant: NVIDIA Corporation

Inventor： Zongyi Yang ,  Mariusz Bojarski ,  Bernhard Firner

IPC: G06V20/56 ,  G06V10/774 ,  G06V10/94 ,  G06T7/73 ,  G06V10/82

CPC classification number: G06V20/588 ,  G06V10/774 ,  G06V10/945 ,  G06T7/73 ,  G06V10/82 ,  G06T2207/30256 ,  G06T2207/20081 ,  G06T2207/20092 ,  G06T2207/30241 ,  G06T2207/20084

Abstract: In various examples, image space coordinates of an image from a video may be labeled, projected to determine 3D vehicle space coordinates, then transformed to 3D world space coordinates using known 3D world space coordinates and relative positioning between the coordinate spaces. For example, 3D vehicle space coordinates may be temporally correlated with known 3D world space coordinates measured while capturing the video. The known 3D world space coordinates and known relative positioning between the coordinate spaces may be used to offset or otherwise define a transform for the 3D vehicle space coordinates to world space. Resultant 3D world space coordinates may be used for one or more labeled frames to generate ground truth data. For example, 3D world space coordinates for left and right lane lines from multiple frames may be used to define lane lines for any given frame.

公开(公告)号：US20200339109A1

公开(公告)日：2020-10-29

申请号：US16860824

申请日：2020-04-28

Applicant: NVIDIA Corporation

Inventor： Jesse Hong ,  Urs Muller ,  Bernhard Firner ,  Zongyi Yang ,  Joyjit Daw ,  David Nister ,  Roberto Giuseppe Luca Valenti ,  Rotem Aviv

IPC: B60W30/08 ,  B60W30/12 ,  B60W30/14 ,  B60W60/00

Abstract: In various examples, sensor data recorded in the real-world may be leveraged to generate transformed, additional, sensor data to test one or more functions of a vehicle—such as a function of an AEB, CMW, LDW, ALC, or ACC system. Sensor data recorded by the sensors may be augmented, transformed, or otherwise updated to represent sensor data corresponding to state information defined by a simulation test profile for testing the vehicle function(s). Once a set of test data has been generated, the test data may be processed by a system of the vehicle to determine the efficacy of the system with respect to any number of test criteria. As a result, a test set including additional or alternative instances of sensor data may be generated from real-world recorded sensor data to test a vehicle in a variety of test scenarios—including those that may be too dangerous to test in the real-world.

公开(公告)号：US20250069385A1

公开(公告)日：2025-02-27

申请号：US18945136

申请日：2024-11-12

Applicant: NVIDIA Corporation

Inventor： Haiguang Wen ,  Bernhard Firner ,  Mariusz Bojarski ,  Zongyi Yang ,  Urs Muller

IPC: G06V10/82 ,  G06N3/08 ,  G06T7/70 ,  G06T9/00 ,  G06V10/25 ,  G06V10/50 ,  G06V10/52 ,  G06V20/56

Abstract: In examples, image data representative of an image of a field of view of at least one sensor may be received. Source areas may be defined that correspond to a region of the image. Areas and/or dimensions of at least some of the source areas may decrease along at least one direction relative to a perspective of the at least one sensor. A downsampled version of the region (e.g., a downsampled image or feature map of a neural network) may be generated from the source areas based at least in part on mapping the source areas to cells of the downsampled version of the region. Resolutions of the region that are captured by the cells may correspond to the areas of the source areas, such that certain portions of the region (e.g., portions at a far distance from the sensor) retain higher resolution than others.

公开(公告)号：US20240183752A1

公开(公告)日：2024-06-06

申请号：US18442753

申请日：2024-02-15

Applicant: NVIDIA Corporation

Inventor： Jesse Hong ,  Urs Muller ,  Bernhard Firner ,  Zongyi Yang ,  Joyjit Daw ,  David Nister ,  Roberto Giuseppe Luca Valenti ,  Rotem Aviv

IPC: G01M17/007 ,  B60W30/08 ,  B60W30/12 ,  B60W30/14 ,  B60W50/00 ,  B60W50/04 ,  B60W60/00 ,  G06F11/36 ,  G06V10/774 ,  G06V20/56 ,  G07C5/08

CPC classification number: G01M17/007 ,  B60W30/08 ,  B60W30/12 ,  B60W30/143 ,  B60W50/04 ,  B60W50/045 ,  B60W60/0011 ,  G06V10/774 ,  G06V20/56 ,  G07C5/08 ,  B60W2050/0028 ,  G06F11/3684 ,  G06F11/3696

Abstract: In various examples, sensor data recorded in the real-world may be leveraged to generate transformed, additional, sensor data to test one or more functions of a vehicle—such as a function of an AEB, CMW, LDW, ALC, or ACC system. Sensor data recorded by the sensors may be augmented, transformed, or otherwise updated to represent sensor data corresponding to state information defined by a simulation test profile for testing the vehicle function(s). Once a set of test data has been generated, the test data may be processed by a system of the vehicle to determine the efficacy of the system with respect to any number of test criteria. As a result, a test set including additional or alternative instances of sensor data may be generated from real-world recorded sensor data to test a vehicle in a variety of test scenarios.

公开(公告)号：US20230110713A1

公开(公告)日：2023-04-13

申请号：US17497479

申请日：2021-10-08

Applicant: NVIDIA Corporation

Inventor： Alperen Degirmenci ,  Won Hong ,  Mariusz Bojarski ,  Jesper Eduard van Engelen ,  Bernhard Firner ,  Zongyi Yang ,  Urs Muller

IPC: G05D1/02 ,  G06N3/00 ,  B60W60/00

Abstract: In various examples, a plurality of poses corresponding to one or more configuration parameters within an environment—such as a location of a machine within an environment, an orientation of a machine within an environment, a sensor angle pose of a machine, or a sensor location of a machine—may be used to generate training data and corresponding ground truth data for training a machine learning model—such as a deep neural network (DNN). As a result, the machine learning model, once deployed, may more accurately compute one or more outputs—such as outputs representative of lane boundaries, trajectories for an autonomous machine, etc.—agnostic to machine and/or sensor poses of the machine within which the machine learning model is deployed.

公开(公告)号：US20240394337A1

公开(公告)日：2024-11-28

申请号：US18791867

申请日：2024-08-01

Applicant: NVIDIA Corporation

Inventor： Zongyi Yang

IPC: G06F18/214 ,  G06N3/08 ,  G06N20/00 ,  G06T15/20

Abstract: In various examples, sets of testing data may be selected and applied to an MLM such that differences in performance of the MLM in the testing between the sets indicates and may be used to determine whether and/or an extent by which the MLM is trained to rely on artifacts. Training data for the MLM may be generated using a first value of a parameter that defines a value of a characteristic of the training data. For testing, first testing data may be selected that corresponds to a second value of the parameter that shifts the value in a first direction and second testing data may be selected that corresponds to a third value of the parameter that shifts the value in a second direction (e.g., opposite the first direction). Various possible actions may be taken based on results of analyzing the differences in performance.

公开(公告)号：US12086208B2

公开(公告)日：2024-09-10

申请号：US17448249

申请日：2021-09-21

Applicant: NVIDIA Corporation

Inventor： Zongyi Yang

IPC: G06F18/214 ,  G06N3/08 ,  G06N20/00 ,  G06T15/20

CPC classification number: G06F18/214 ,  G06N3/08 ,  G06N20/00 ,  G06T15/20

Abstract: In various examples, sets of testing data may be selected and applied to an MLM such that differences in performance of the MLM in the testing between the sets indicates and may be used to determine whether and/or an extent by which the MLM is trained to rely on artifacts. Training data for the MLM may be generated using a first value of a parameter that defines a value of a characteristic of the training data. For testing, first testing data may be selected that corresponds to a second value of the parameter that shifts the value in a first direction and second testing data may be selected that corresponds to a third value of the parameter that shifts the value in a second direction (e.g., opposite the first direction). Various possible actions may be taken based on results of analyzing the differences in performance.

公开(公告)号：US20220092349A1

公开(公告)日：2022-03-24

申请号：US17448249

申请日：2021-09-21

Applicant: NVIDIA Corporation

Inventor： Zongyi Yang

IPC: G06K9/62 ,  G06N20/00 ,  G06N3/08 ,  G06T15/20

Abstract: In various examples, sets of testing data may be selected and applied to an MLM such that differences in performance of the MLM in the testing between the sets indicates and may be used to determine whether and/or an extent by which the MLM is trained to rely on artifacts. Training data for the MLM may be generated using a first value of a parameter that defines a value of a characteristic of the training data. For testing, first testing data may be selected that corresponds to a second value of the parameter that shifts the value in a first direction and second testing data may be selected that corresponds to a third value of the parameter that shifts the value in a second direction (e.g., opposite the first direction). Various possible actions may be taken based on results of analyzing the differences in performance.

公开(公告)号：US20220092317A1

公开(公告)日：2022-03-24

申请号：US17448247

申请日：2021-09-21

Applicant: NVIDIA Corporation

Inventor： Zongyi Yang ,  Mariusz Bojarski ,  Bernhard Firner ,  Urs Muller

IPC: G06K9/00 ,  G05B13/02 ,  G06K9/32 ,  G06T5/00 ,  G06K9/62 ,  G06T3/60 ,  G06T7/11

Abstract: In various examples, sensor data used to train an MLM and/or used by the MLM during deployment, may be captured by sensors having different perspectives (e.g., fields of view). The sensor data may be transformed—to generate transformed sensor data—such as by altering or removing lens distortions, shifting, and/or rotating images corresponding to the sensor data to a field of view of a different physical or virtual sensor. As such, the MLM may be trained and/or deployed using sensor data captured from a same or similar field of view. As a result, the MLM may be trained and/or deployed—across any number of different vehicles with cameras and/or other sensors having different perspectives—using sensor data that is of the same perspective as the reference or ideal sensor.

公开(公告)号：US12183063B2

公开(公告)日：2024-12-31

申请号：US16917289

申请日：2020-06-30

Applicant: NVIDIA Corporation

Inventor： Haiguang Wen ,  Bernhard Firner ,  Mariusz Bojarski ,  Zongyi Yang ,  Urs Muller

IPC: G06K9/00 ,  G06N3/08 ,  G06T7/70 ,  G06T9/00 ,  G06V10/25 ,  G06V10/50 ,  G06V10/52 ,  G06V10/82 ,  G06V20/56

Abstract: In examples, image data representative of an image of a field of view of at least one sensor may be received. Source areas may be defined that correspond to a region of the image. Areas and/or dimensions of at least some of the source areas may decrease along at least one direction relative to a perspective of the at least one sensor. A downsampled version of the region (e.g., a downsampled image or feature map of a neural network) may be generated from the source areas based at least in part on mapping the source areas to cells of the downsampled version of the region. Resolutions of the region that are captured by the cells may correspond to the areas of the source areas, such that certain portions of the region (e.g., portions at a far distance from the sensor) retain higher resolution than others.