公开(公告)号：US20230213945A1

公开(公告)日：2023-07-06

申请号：US17565837

申请日：2021-12-30

Applicant: NVIDIA Corporation

Inventor： Neeraj Sajjan ,  Mehmet K. Kocamaz ,  Junghyun Kwon ,  Sangmin Oh ,  Minwoo Park ,  David Nister

IPC: G05D1/02 ,  G06N3/08 ,  G05D1/00

CPC classification number: G05D1/0248 ,  G05D1/0257 ,  G06N3/08 ,  G05D1/0221 ,  G05D1/0219 ,  G05D1/0088 ,  G05D1/0251 ,  G05D1/0255 ,  G05D2201/0213

Abstract: In various examples, one or more output channels of a deep neural network (DNN) may be used to determine assignments of obstacles to paths. To increase the accuracy of the DNN, the input to the DNN may include an input image, one or more representations of path locations, and/or one or more representations of obstacle locations. The system may thus repurpose previously computed information—e.g., obstacle locations, path locations, etc.—from other operations of the system, and use them to generate more detailed inputs for the DNN to increase accuracy of the obstacle to path assignments. Once the output channels are computed using the DNN, computed bounding shapes for the objects may be compared to the outputs to determine the path assignments for each object.

公开(公告)号：US12272152B2

公开(公告)日：2025-04-08

申请号：US17551986

申请日：2021-12-15

Applicant: NVIDIA Corporation

Inventor： Mehmet K. Kocamaz ,  Ke Xu ,  Sangmin Oh ,  Junghyun Kwon

IPC: G06V20/58 ,  B60W60/00 ,  G06F18/213 ,  G06N3/08 ,  G06T7/246 ,  G06V10/22 ,  G06V10/46 ,  G06V10/82

Abstract: In various examples, live perception from sensors of a vehicle may be leveraged to generate object tracking paths for the vehicle to facilitate navigational controls in real-time or near real-time. For example, a deep neural network (DNN) may be trained to compute various outputs—such as feature descriptor maps including feature descriptor vectors corresponding to objects included in a sensor(s) field of view. The outputs may be decoded and/or otherwise post-processed to reconstruct object tracking and to determine proposed or potential paths for navigating the vehicle.

公开(公告)号：US20230360255A1

公开(公告)日：2023-11-09

申请号：US17955822

申请日：2022-09-29

Applicant: NVIDIA Corporation

Inventor： Mehmet K. Kocamaz ,  Daniel Per Olof Svensson ,  Hang Dou ,  Sangmin Oh ,  Minwoo Park ,  Kexuan Zou

IPC: G06T7/73 ,  G06T7/20

CPC classification number: G06T7/73 ,  G06T7/20 ,  G06V2201/07 ,  G06T2207/30241 ,  G06T2207/30252

Abstract: In various examples, techniques for multi-dimensional tracking of objects using two-dimensional (2D) sensor data are described. Systems and methods may use first image data to determine a first 2D detected location and a first three-dimensional (3D) detected location of an object. The systems and methods may then determine a 2D estimated location using the first 2D detected location and a 3D estimated location using the first 3D detected location. The systems and methods may use second image data to determine a second 2D detected location and a second 3D detected location of a detected object, and may then determine that the object corresponds to the detected object using the 2D estimated location, the 3D estimated location, the second 2D detected location, and the second 3D detected location. The systems and method then generate, modify, delete, or otherwise update an object track that includes 2D state information and 3D state information.

公开(公告)号：US20230360232A1

公开(公告)日：2023-11-09

申请号：US17955827

申请日：2022-09-29

Applicant: NVIDIA Corporation

Inventor： Mehmet K. Kocamaz ,  Parthiv Parikh ,  Sangmin Oh

IPC: G06T7/246

CPC classification number: G06T7/248 ,  G06T2207/30261

Abstract: In various examples, systems and methods for tracking objects and determining time-to-collision values associated with the objects are described. For instance, the systems and methods may use feature points associated with an object depicted in a first image and feature points associated with a second image to determine a scalar change associated with the object. The systems and methods may then use the scalar change to determine a translation associated with the object. Using the scalar change and the translation, the systems and methods may determine that the object is also depicted in the second image. The systems and methods may further use the scalar change and a temporal baseline to determine a time-to-collision associated with the object. After performing the determinations, the systems and methods may output data representing at least an identifier for the object, a location of the object, and/or the time-to-collision.

公开(公告)号：US20210406560A1

公开(公告)日：2021-12-30

申请号：US17353231

申请日：2021-06-21

Applicant: NVIDIA Corporation

Inventor： Minwoo Park ,  Junghyun Kwon ,  Mehmet K. Kocamaz ,  Hae-Jong Seo ,  Berta Rodriguez Hervas ,  Tae Eun Choe

IPC: G06K9/00 ,  B60W60/00 ,  G06T7/292

Abstract: In various examples, a multi-sensor fusion machine learning model—such as a deep neural network (DNN)—may be deployed to fuse data from a plurality of individual machine learning models. As such, the multi-sensor fusion network may use outputs from a plurality of machine learning models as input to generate a fused output that represents data from fields of view or sensory fields of each of the sensors supplying the machine learning models, while accounting for learned associations between boundary or overlap regions of the various fields of view of the source sensors. In this way, the fused output may be less likely to include duplicate, inaccurate, or noisy data with respect to objects or features in the environment, as the fusion network may be trained to account for multiple instances of a same object appearing in different input representations.

公开(公告)号：US20230282005A1

公开(公告)日：2023-09-07

申请号：US18309878

申请日：2023-05-01

Applicant: NVIDIA Corporation

Inventor： Minwoo Park ,  Junghyun Kwon ,  Mehmet K. Kocamaz ,  Hae-Jong Seo ,  Berta Rodriguez Hervas ,  Tae Eun Choe

IPC: G06V20/56 ,  B60W60/00 ,  G06T7/292 ,  G06V20/58

CPC classification number: G06V20/588 ,  B60W60/00272 ,  G06T7/292 ,  G06V20/58 ,  B60W2554/4029 ,  B60W2554/4044 ,  B60W2556/35 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: In various examples, a multi-sensor fusion machine learning model – such as a deep neural network (DNN) – may be deployed to fuse data from a plurality of individual machine learning models. As such, the multi-sensor fusion network may use outputs from a plurality of machine learning models as input to generate a fused output that represents data from fields of view or sensory fields of each of the sensors supplying the machine learning models, while accounting for learned associations between boundary or overlap regions of the various fields of view of the source sensors. In this way, the fused output may be less likely to include duplicate, inaccurate, or noisy data with respect to objects or features in the environment, as the fusion network may be trained to account for multiple instances of a same object appearing in different input representations.

公开(公告)号：US11688181B2

公开(公告)日：2023-06-27

申请号：US17353231

申请日：2021-06-21

Applicant: NVIDIA Corporation

Inventor： Minwoo Park ,  Junghyun Kwon ,  Mehmet K. Kocamaz ,  Hae-Jong Seo ,  Berta Rodriguez Hervas ,  Tae Eun Choe

IPC: G06V20/58 ,  G06V20/56 ,  B60W60/00 ,  G06T7/292

CPC classification number: G06V20/588 ,  B60W60/00272 ,  G06T7/292 ,  G06V20/58 ,  B60W2554/4029 ,  B60W2554/4044 ,  B60W2556/35 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: In various examples, a multi-sensor fusion machine learning model—such as a deep neural network (DNN)—may be deployed to fuse data from a plurality of individual machine learning models. As such, the multi-sensor fusion network may use outputs from a plurality of machine learning models as input to generate a fused output that represents data from fields of view or sensory fields of each of the sensors supplying the machine learning models, while accounting for learned associations between boundary or overlap regions of the various fields of view of the source sensors. In this way, the fused output may be less likely to include duplicate, inaccurate, or noisy data with respect to objects or features in the environment, as the fusion network may be trained to account for multiple instances of a same object appearing in different input representations.

公开(公告)号：US20240428596A1

公开(公告)日：2024-12-26

申请号：US18074708

申请日：2022-12-05

Applicant: NVIDIA Corporation

Inventor： Parthiv Parikh ,  Mehmet K. Kocamaz

IPC: G06V20/58

Abstract: In various examples, object track management for autonomous or semi-autonomous systems and applications is described herein. Systems and methods are disclosed that may limit the number of objects that are tracked based on one or more criteria. For instance, the number of objects that are tracked may be limited to a threshold number of objects when a number of detected objects exceeds a threshold. The systems and methods may use parameters associated with the detected objects to determine priority scores associated with the detected objects, and may then determine to only track the detected objects with the highest scores (e.g., high priority objects). As a result, latency and compute of the system may be reduced while still maintaining tracking with respect to safety-critical objects.

公开(公告)号：US20230360231A1

公开(公告)日：2023-11-09

申请号：US17955814

申请日：2022-09-29

Applicant: NVIDIA Corporation

Inventor： Mehmet K. Kocamaz ,  Daniel Per Olof Svensson ,  Hang Dou ,  Sangmin Oh ,  Minwoo Park ,  Kexuan Zou

IPC: G06T7/246

CPC classification number: G06T7/246 ,  G06T2207/30252

Abstract: In various examples, techniques for multi-dimensional tracking of objects using two-dimensional (2D) sensor data are described. Systems and methods may use first image data to determine a first 2D detected location and a first three-dimensional (3D) detected location of an object. The systems and methods may then determine a 2D estimated location using the first 2D detected location and a 3D estimated location using the first 3D detected location. The systems and methods may use second image data to determine a second 2D detected location and a second 3D detected location of a detected object, and may then determine that the object corresponds to the detected object using the 2D estimated location, the 3D estimated location, the second 2D detected location, and the second 3D detected location. The systems and method then generate, modify, delete, or otherwise update an object track that includes 2D state information and 3D state information.

公开(公告)号：US20230186640A1

公开(公告)日：2023-06-15

申请号：US17551986

申请日：2021-12-15

Applicant: NVIDIA Corporation

Inventor： Mehmet K. Kocamaz ,  Ke Xu ,  Sangmin Oh ,  Junghyun Kwon

IPC: G06V20/58 ,  G06K9/62 ,  G06V10/82 ,  G06V10/46 ,  G06V10/22 ,  G06T7/246 ,  B60W60/00 ,  G06N3/08

CPC classification number: G06V20/58 ,  G06K9/6232 ,  G06V10/82 ,  G06V10/46 ,  G06V10/225 ,  G06T7/246 ,  B60W60/001 ,  G06N3/08 ,  G06T2207/30252 ,  G06T2207/20084 ,  G06T2207/20081 ,  B60W2420/42

Abstract: In various examples, live perception from sensors of a vehicle may be leveraged to generate object tracking paths for the vehicle to facilitate navigational controls in real-time or near real-time. For example, a deep neural network (DNN) may be trained to compute various outputs—such as feature descriptor maps including feature descriptor vectors corresponding to objects included in a sensor(s) field of view. The outputs may be decoded and/or otherwise post-processed to reconstruct object tracking and to determine proposed or potential paths for navigating the vehicle.