公开(公告)号：US20230142299A1

公开(公告)日：2023-05-11

申请号：US17454389

申请日：2021-11-10

申请人： NVIDIA Corporation

发明人： Gang Pan ,  Joachim Pehserl ,  Dong Zhang ,  Baris Evrim Demiroz ,  Samuel Rupp Ogden ,  Tae Eun Choe ,  Sangmin Oh

IPC分类号： G01S13/931 ,  G01S13/86 ,  G01S17/931 ,  G01S17/86

CPC分类号： G01S13/931 ,  G01S13/865 ,  G01S17/931 ,  G01S17/86 ,  G01S13/867 ,  G01S2013/932 ,  G01S2013/9318

摘要： In various examples, a hazard detection system fuses outputs from multiple sensors over time to determine a probability that a stationary object or hazard exists at a location. The system may then use sensor data to calculate a detection bounding shape for detected objects and, using the bounding shape, may generate a set of particles, each including a confidence value that an object exists at a corresponding location. The system may then capture additional sensor data by one or more sensors of the ego-machine that are different from those used to capture the first sensor data. To improve the accuracy of the confidences of the particles, the system may determine a correspondence between the first sensor data and the additional sensor data (e.g., depth sensor data), which may be used to filter out a portion of the particles and improve the depth predictions corresponding to the object.

公开(公告)号：US20210309248A1

公开(公告)日：2021-10-07

申请号：US17150954

申请日：2021-01-15

申请人： NVIDIA Corporation

发明人： Tae Eun Choe ,  Pengfei Hao ,  Xiaolin Lin ,  Minwoo Park

IPC分类号： B60W60/00 ,  G06N3/04 ,  G06N3/08 ,  B60W50/06

摘要： In various examples, systems and methods are disclosed that preserve rich, detail-centric information from a real-world image by augmenting the real-world image with simulated objects to train a machine learning model to detect objects in an input image. The machine learning model may be trained, in deployment, to detect objects and determine bounding shapes to encapsulate detected objects. The machine learning model may further be trained to determine the type of road object encountered, calculate hazard ratings, and calculate confidence percentages. In deployment, detection of a road object, determination of a corresponding bounding shape, identification of road object type, and/or calculation of a hazard rating by the machine learning model may be used as an aid for determining next steps regarding the surrounding environment—e.g., navigating around the road debris, driving over the road debris, or coming to a complete stop—in a variety of autonomous machine applications.

公开(公告)号：US20240320923A1

公开(公告)日：2024-09-26

申请号：US18680454

申请日：2024-05-31

申请人： NVIDIA CORPORATION

发明人： Ahyun SEO ,  Tae Eun Choe ,  Minwoo Park ,  Jung Seock Joo

IPC分类号： G06T19/00 ,  H04N13/111 ,  H04N13/282

CPC分类号： G06T19/003 ,  H04N13/111 ,  H04N13/282

摘要： Systems and methods are disclosed relating to viewpoint adapted perception for autonomous machines and applications. A 3D perception network may be adapted to handle unavailable target rig data by training the one or more layers of the 3D perception network as part of a training network using simulated source and target rig data. A consistency loss that compares (e.g., top-down) transformed feature maps extracted from simulated source and target rig data may be used to minimize differences across training channels. As such, one or more of the paths through the training network(s) may be designated as the 3D perception network, and target rig data may be applied to the 3D perception network to perform one or more perception tasks.

公开(公告)号：US20240265555A1

公开(公告)日：2024-08-08

申请号：US18614160

申请日：2024-03-22

申请人： NVIDIA Corporation

发明人： Dong Zhang ,  Sangmin Oh ,  Junghyun Kwon ,  Baris Evrim Demiroz ,  Tae Eun Choe ,  Minwoo Park ,  Chethan Ningaraju ,  Hao Tsui ,  Eric Viscito ,  Jagadeesh Sankaran ,  Yongqing Liang

IPC分类号： G06T7/246 ,  B60W60/00 ,  G06F18/214 ,  G06N3/08 ,  G06V10/25 ,  G06V10/75 ,  G06V20/56 ,  G06V20/58

CPC分类号： G06T7/246 ,  B60W60/001 ,  G06F18/2148 ,  G06N3/08 ,  G06V10/25 ,  G06V10/751 ,  G06V20/58 ,  G06V20/56

摘要： Systems and methods are disclosed that use a geometric approach to detect objects on a road surface. A set of points within a region of interest between a first frame and a second frame are captured and tracked to determine a difference in location between the set of points in two frames. The first frame may be aligned with the second frame and the first pixel values of the first frame may be compared with the second pixel values of the second frame to generate a disparity image including third pixels. Subsets of the third pixels that have an disparity image value about a first threshold may be combined, and the third pixels may be scored and associated with disparity values for each pixel of the one or more subsets of the third pixels. A bounding shape may be generated based on the scoring that corresponds to the object.

公开(公告)号：US20220122001A1

公开(公告)日：2022-04-21

申请号：US17219350

申请日：2021-03-31

申请人： Nvidia Corporation

发明人： Tae Eun Choe ,  Aman Kishore ,  Junghyun Kwon ,  Minwoo Park ,  Pengfei Hao ,  Akshita Mittel

IPC分类号： G06N20/20 ,  A63F13/50 ,  G06N3/04

摘要： Approaches presented herein provide for the generation of synthetic data to fortify a dataset for use in training a network via imitation learning. In at least one embodiment, a system is evaluated to identify failure cases, such as may correspond to false positives and false negative detections. Additional synthetic data imitating these failure cases can then be generated and utilized to provide a more abundant dataset. A network or model can then be trained, or retrained, with the original training data and the additional synthetic data. In one or more embodiments, these steps may be repeated until the evaluation metric converges, with additional synthetic training data being generated corresponding to the failure cases at each training pass.

公开(公告)号：US20240312123A1

公开(公告)日：2024-09-19

申请号：US18592025

申请日：2024-02-29

申请人： NVIDIA Corporation

发明人： Malik Aqeel Anwar ,  Tae Eun Choe ,  Zian Wang ,  Sanja Fidler ,  Minwoo Park

IPC分类号： G06T15/50 ,  G06T15/60 ,  G06T19/20 ,  G06V10/774 ,  H04N23/698

CPC分类号： G06T15/506 ,  G06T15/60 ,  G06T19/20 ,  H04N23/698 ,  G06T2219/2004 ,  G06V10/774

摘要： In various examples, systems and methods are disclosed that relate to data augmentation for training/updating perception models in autonomous or semi-autonomous systems and applications. For example, a system may receive data associated with a set of frames that are captured using a plurality of cameras positioned in fixed relation relative to the machine; generate a panoramic view based at least on the set of frames; provide data associated with the panoramic view to a model to cause the model to generate a high dynamic range (HDR) panoramic view; determine lighting information associated with a light distribution map based at least on the HDR panoramic view; determine a virtual scene; and render an asset and a shadow on at least one of the frames, based at least on the virtual scene and the light distribution map, the shadow being a shadow corresponding to the asset.

公开(公告)号：US11961243B2

公开(公告)日：2024-04-16

申请号：US17187228

申请日：2021-02-26

申请人： NVIDIA Corporation

发明人： Dong Zhang ,  Sangmin Oh ,  Junghyun Kwon ,  Baris Evrim Demiroz ,  Tae Eun Choe ,  Minwoo Park ,  Chethan Ningaraju ,  Hao Tsui ,  Eric Viscito ,  Jagadeesh Sankaran ,  Yongqing Liang

IPC分类号： G06T7/00 ,  B60W60/00 ,  G06F18/214 ,  G06N3/08 ,  G06T7/246 ,  G06V10/25 ,  G06V10/75 ,  G06V20/58 ,  G06V20/56

CPC分类号： G06T7/246 ,  B60W60/001 ,  G06F18/2148 ,  G06N3/08 ,  G06V10/25 ,  G06V10/751 ,  G06V20/58 ,  G06V20/56

摘要： A geometric approach may be used to detect objects on a road surface. A set of points within a region of interest between a first frame and a second frame are captured and tracked to determine a difference in location between the set of points in two frames. The first frame may be aligned with the second frame and the first pixel values of the first frame may be compared with the second pixel values of the second frame to generate a disparity image including third pixels. One or more subsets of the third pixels that have a value above a first threshold may be combined, and the third pixels may be scored and associated with disparity values for each pixel of the one or more subsets of the third pixels. A bounding shape may be generated based on the scoring.

公开(公告)号：US20240001957A1

公开(公告)日：2024-01-04

申请号：US18467123

申请日：2023-09-14

申请人： NVIDIA Corporation

发明人： Tae Eun Choe ,  Pengfei Hao ,  Xiaolin Lin ,  Minwoo Park

IPC分类号： B60W60/00 ,  G06N3/04 ,  G06N3/08 ,  B60W50/06

CPC分类号： B60W60/001 ,  G06N3/04 ,  G06N3/08 ,  B60W50/06 ,  B60W2554/80 ,  B60W2420/42 ,  B60W2554/4029

摘要： In various examples, systems and methods are disclosed that preserve rich, detail-centric information from a real-world image by augmenting the real-world image with simulated objects to train a machine learning model to detect objects in an input image. The machine learning model may be trained, in deployment, to detect objects and determine bounding shapes to encapsulate detected objects. The machine learning model may further be trained to determine the type of road object encountered, calculate hazard ratings, and calculate confidence percentages. In deployment, detection of a road object, determination of a corresponding bounding shape, identification of road object type, and/or calculation of a hazard rating by the machine learning model may be used as an aid for determining next steps regarding the surrounding environment—e.g., navigating around the road debris, driving over the road debris, or coming to a complete stop—in a variety of autonomous machine applications.

公开(公告)号：US20230282005A1

公开(公告)日：2023-09-07

申请号：US18309878

申请日：2023-05-01

申请人： NVIDIA Corporation

发明人： Minwoo Park ,  Junghyun Kwon ,  Mehmet K. Kocamaz ,  Hae-Jong Seo ,  Berta Rodriguez Hervas ,  Tae Eun Choe

IPC分类号： G06V20/56 ,  B60W60/00 ,  G06T7/292 ,  G06V20/58

CPC分类号： G06V20/588 ,  B60W60/00272 ,  G06T7/292 ,  G06V20/58 ,  B60W2554/4029 ,  B60W2554/4044 ,  B60W2556/35 ,  G06T2207/20081 ,  G06T2207/20084

摘要： In various examples, a multi-sensor fusion machine learning model – such as a deep neural network (DNN) – may be deployed to fuse data from a plurality of individual machine learning models. As such, the multi-sensor fusion network may use outputs from a plurality of machine learning models as input to generate a fused output that represents data from fields of view or sensory fields of each of the sensors supplying the machine learning models, while accounting for learned associations between boundary or overlap regions of the various fields of view of the source sensors. In this way, the fused output may be less likely to include duplicate, inaccurate, or noisy data with respect to objects or features in the environment, as the fusion network may be trained to account for multiple instances of a same object appearing in different input representations.

公开(公告)号：US11688181B2

公开(公告)日：2023-06-27

申请号：US17353231

申请日：2021-06-21

申请人： NVIDIA Corporation

发明人： Minwoo Park ,  Junghyun Kwon ,  Mehmet K. Kocamaz ,  Hae-Jong Seo ,  Berta Rodriguez Hervas ,  Tae Eun Choe

IPC分类号： G06V20/58 ,  G06V20/56 ,  B60W60/00 ,  G06T7/292

CPC分类号： G06V20/588 ,  B60W60/00272 ,  G06T7/292 ,  G06V20/58 ,  B60W2554/4029 ,  B60W2554/4044 ,  B60W2556/35 ,  G06T2207/20081 ,  G06T2207/20084

摘要： In various examples, a multi-sensor fusion machine learning model—such as a deep neural network (DNN)—may be deployed to fuse data from a plurality of individual machine learning models. As such, the multi-sensor fusion network may use outputs from a plurality of machine learning models as input to generate a fused output that represents data from fields of view or sensory fields of each of the sensors supplying the machine learning models, while accounting for learned associations between boundary or overlap regions of the various fields of view of the source sensors. In this way, the fused output may be less likely to include duplicate, inaccurate, or noisy data with respect to objects or features in the environment, as the fusion network may be trained to account for multiple instances of a same object appearing in different input representations.