公开(公告)号：US20230360255A1

公开(公告)日：2023-11-09

申请号：US17955822

申请日：2022-09-29

申请人： NVIDIA Corporation

发明人： Mehmet K. Kocamaz ,  Daniel Per Olof Svensson ,  Hang Dou ,  Sangmin Oh ,  Minwoo Park ,  Kexuan Zou

IPC分类号： G06T7/73 ,  G06T7/20

CPC分类号： G06T7/73 ,  G06T7/20 ,  G06V2201/07 ,  G06T2207/30241 ,  G06T2207/30252

摘要： In various examples, techniques for multi-dimensional tracking of objects using two-dimensional (2D) sensor data are described. Systems and methods may use first image data to determine a first 2D detected location and a first three-dimensional (3D) detected location of an object. The systems and methods may then determine a 2D estimated location using the first 2D detected location and a 3D estimated location using the first 3D detected location. The systems and methods may use second image data to determine a second 2D detected location and a second 3D detected location of a detected object, and may then determine that the object corresponds to the detected object using the 2D estimated location, the 3D estimated location, the second 2D detected location, and the second 3D detected location. The systems and method then generate, modify, delete, or otherwise update an object track that includes 2D state information and 3D state information.

公开(公告)号：US20230213945A1

公开(公告)日：2023-07-06

申请号：US17565837

申请日：2021-12-30

申请人： NVIDIA Corporation

发明人： Neeraj Sajjan ,  Mehmet K. Kocamaz ,  Junghyun Kwon ,  Sangmin Oh ,  Minwoo Park ,  David Nister

IPC分类号： G05D1/02 ,  G06N3/08 ,  G05D1/00

CPC分类号： G05D1/0248 ,  G05D1/0257 ,  G06N3/08 ,  G05D1/0221 ,  G05D1/0219 ,  G05D1/0088 ,  G05D1/0251 ,  G05D1/0255 ,  G05D2201/0213

摘要： In various examples, one or more output channels of a deep neural network (DNN) may be used to determine assignments of obstacles to paths. To increase the accuracy of the DNN, the input to the DNN may include an input image, one or more representations of path locations, and/or one or more representations of obstacle locations. The system may thus repurpose previously computed information—e.g., obstacle locations, path locations, etc.—from other operations of the system, and use them to generate more detailed inputs for the DNN to increase accuracy of the obstacle to path assignments. Once the output channels are computed using the DNN, computed bounding shapes for the objects may be compared to the outputs to determine the path assignments for each object.

公开(公告)号：US20230282005A1

公开(公告)日：2023-09-07

申请号：US18309878

申请日：2023-05-01

申请人： NVIDIA Corporation

发明人： Minwoo Park ,  Junghyun Kwon ,  Mehmet K. Kocamaz ,  Hae-Jong Seo ,  Berta Rodriguez Hervas ,  Tae Eun Choe

IPC分类号： G06V20/56 ,  B60W60/00 ,  G06T7/292 ,  G06V20/58

CPC分类号： G06V20/588 ,  B60W60/00272 ,  G06T7/292 ,  G06V20/58 ,  B60W2554/4029 ,  B60W2554/4044 ,  B60W2556/35 ,  G06T2207/20081 ,  G06T2207/20084

摘要： In various examples, a multi-sensor fusion machine learning model – such as a deep neural network (DNN) – may be deployed to fuse data from a plurality of individual machine learning models. As such, the multi-sensor fusion network may use outputs from a plurality of machine learning models as input to generate a fused output that represents data from fields of view or sensory fields of each of the sensors supplying the machine learning models, while accounting for learned associations between boundary or overlap regions of the various fields of view of the source sensors. In this way, the fused output may be less likely to include duplicate, inaccurate, or noisy data with respect to objects or features in the environment, as the fusion network may be trained to account for multiple instances of a same object appearing in different input representations.

公开(公告)号：US11688181B2

公开(公告)日：2023-06-27

申请号：US17353231

申请日：2021-06-21

申请人： NVIDIA Corporation

发明人： Minwoo Park ,  Junghyun Kwon ,  Mehmet K. Kocamaz ,  Hae-Jong Seo ,  Berta Rodriguez Hervas ,  Tae Eun Choe

IPC分类号： G06V20/58 ,  G06V20/56 ,  B60W60/00 ,  G06T7/292

CPC分类号： G06V20/588 ,  B60W60/00272 ,  G06T7/292 ,  G06V20/58 ,  B60W2554/4029 ,  B60W2554/4044 ,  B60W2556/35 ,  G06T2207/20081 ,  G06T2207/20084

摘要： In various examples, a multi-sensor fusion machine learning model—such as a deep neural network (DNN)—may be deployed to fuse data from a plurality of individual machine learning models. As such, the multi-sensor fusion network may use outputs from a plurality of machine learning models as input to generate a fused output that represents data from fields of view or sensory fields of each of the sensors supplying the machine learning models, while accounting for learned associations between boundary or overlap regions of the various fields of view of the source sensors. In this way, the fused output may be less likely to include duplicate, inaccurate, or noisy data with respect to objects or features in the environment, as the fusion network may be trained to account for multiple instances of a same object appearing in different input representations.

公开(公告)号：US20230360232A1

公开(公告)日：2023-11-09

申请号：US17955827

申请日：2022-09-29

申请人： NVIDIA Corporation

发明人： Mehmet K. Kocamaz ,  Parthiv Parikh ,  Sangmin Oh

IPC分类号： G06T7/246

CPC分类号： G06T7/248 ,  G06T2207/30261

摘要： In various examples, systems and methods for tracking objects and determining time-to-collision values associated with the objects are described. For instance, the systems and methods may use feature points associated with an object depicted in a first image and feature points associated with a second image to determine a scalar change associated with the object. The systems and methods may then use the scalar change to determine a translation associated with the object. Using the scalar change and the translation, the systems and methods may determine that the object is also depicted in the second image. The systems and methods may further use the scalar change and a temporal baseline to determine a time-to-collision associated with the object. After performing the determinations, the systems and methods may output data representing at least an identifier for the object, a location of the object, and/or the time-to-collision.

公开(公告)号：US20210406560A1

公开(公告)日：2021-12-30

申请号：US17353231

申请日：2021-06-21

申请人： NVIDIA Corporation

发明人： Minwoo Park ,  Junghyun Kwon ,  Mehmet K. Kocamaz ,  Hae-Jong Seo ,  Berta Rodriguez Hervas ,  Tae Eun Choe

IPC分类号： G06K9/00 ,  B60W60/00 ,  G06T7/292

摘要： In various examples, a multi-sensor fusion machine learning model—such as a deep neural network (DNN)—may be deployed to fuse data from a plurality of individual machine learning models. As such, the multi-sensor fusion network may use outputs from a plurality of machine learning models as input to generate a fused output that represents data from fields of view or sensory fields of each of the sensors supplying the machine learning models, while accounting for learned associations between boundary or overlap regions of the various fields of view of the source sensors. In this way, the fused output may be less likely to include duplicate, inaccurate, or noisy data with respect to objects or features in the environment, as the fusion network may be trained to account for multiple instances of a same object appearing in different input representations.

公开(公告)号：US20230360231A1

公开(公告)日：2023-11-09

申请号：US17955814

申请日：2022-09-29

申请人： NVIDIA Corporation

发明人： Mehmet K. Kocamaz ,  Daniel Per Olof Svensson ,  Hang Dou ,  Sangmin Oh ,  Minwoo Park ,  Kexuan Zou

IPC分类号： G06T7/246

CPC分类号： G06T7/246 ,  G06T2207/30252

摘要： In various examples, techniques for multi-dimensional tracking of objects using two-dimensional (2D) sensor data are described. Systems and methods may use first image data to determine a first 2D detected location and a first three-dimensional (3D) detected location of an object. The systems and methods may then determine a 2D estimated location using the first 2D detected location and a 3D estimated location using the first 3D detected location. The systems and methods may use second image data to determine a second 2D detected location and a second 3D detected location of a detected object, and may then determine that the object corresponds to the detected object using the 2D estimated location, the 3D estimated location, the second 2D detected location, and the second 3D detected location. The systems and method then generate, modify, delete, or otherwise update an object track that includes 2D state information and 3D state information.

公开(公告)号：US20230186640A1

公开(公告)日：2023-06-15

申请号：US17551986

申请日：2021-12-15

申请人： NVIDIA Corporation

发明人： Mehmet K. Kocamaz ,  Ke Xu ,  Sangmin Oh ,  Junghyun Kwon

IPC分类号： G06V20/58 ,  G06K9/62 ,  G06V10/82 ,  G06V10/46 ,  G06V10/22 ,  G06T7/246 ,  B60W60/00 ,  G06N3/08

CPC分类号： G06V20/58 ,  G06K9/6232 ,  G06V10/82 ,  G06V10/46 ,  G06V10/225 ,  G06T7/246 ,  B60W60/001 ,  G06N3/08 ,  G06T2207/30252 ,  G06T2207/20084 ,  G06T2207/20081 ,  B60W2420/42

摘要： In various examples, live perception from sensors of a vehicle may be leveraged to generate object tracking paths for the vehicle to facilitate navigational controls in real-time or near real-time. For example, a deep neural network (DNN) may be trained to compute various outputs—such as feature descriptor maps including feature descriptor vectors corresponding to objects included in a sensor(s) field of view. The outputs may be decoded and/or otherwise post-processed to reconstruct object tracking and to determine proposed or potential paths for navigating the vehicle.