公开(公告)号：US12286115B2

公开(公告)日：2025-04-29

申请号：US17116138

申请日：2020-12-09

Applicant: NVIDIA Corporation

Inventor： Trung Pham ,  Berta Rodriguez Hervas ,  Minwoo Park ,  David Nister ,  Neda Cvijetic

IPC: B60W30/18 ,  G06N3/04 ,  G06N3/08 ,  G06T7/33 ,  G06V10/764 ,  G06V10/82 ,  G06V20/56 ,  G06V20/64 ,  G06N3/045

Abstract: In various examples, a three-dimensional (3D) intersection structure may be predicted using a deep neural network (DNN) based on processing two-dimensional (2D) input data. To train the DNN to accurately predict 3D intersection structures from 2D inputs, the DNN may be trained using a first loss function that compares 3D outputs of the DNN—after conversion to 2D space—to 2D ground truth data and a second loss function that analyzes the 3D predictions of the DNN in view of one or more geometric constraints—e.g., geometric knowledge of intersections may be used to penalize predictions of the DNN that do not align with known intersection and/or road structure geometries. As such, live perception of an autonomous or semi-autonomous vehicle may be used by the DNN to detect 3D locations of intersection structures from 2D inputs.

公开(公告)号：US20250091607A1

公开(公告)日：2025-03-20

申请号：US18971085

申请日：2024-12-06

Applicant: NVIDIA CORPORATION

Inventor： Kang Wang ,  Yue Wu ,  Minwoo Park ,  Gang Pan

IPC: B60W60/00 ,  B60W40/09 ,  G06N3/02

Abstract: In various examples, a 3D surface structure such as the 3D surface structure of a road (3D road surface) may be observed and estimated to generate a 3D point cloud or other representation of the 3D surface structure. Since the estimated representation may be sparse, a deep neural network (DNN) may be used to predict values for a dense representation of the 3D surface structure from the sparse representation. For example, a sparse 3D point cloud may be projected to form a sparse projection image (e.g., a sparse 2D height map), which may be fed into the DNN to predict a dense projection image (e.g., a dense 2D height map). The predicted dense representation of the 3D surface structure may be provided to an autonomous vehicle drive stack to enable safe and comfortable planning and control of the autonomous vehicle.

公开(公告)号：US12248319B2

公开(公告)日：2025-03-11

申请号：US18340255

申请日：2023-06-23

Applicant: NVIDIA Corporation

Inventor： Minwoo Park ,  Xiaolin Lin ,  Hae-Jong Seo ,  David Nister ,  Neda Cvijetic

IPC: G05D1/00 ,  G05D1/228 ,  G06F18/214 ,  G06F18/23 ,  G06F18/2411 ,  G06N3/04 ,  G06N3/045 ,  G06N3/08 ,  G06V10/14 ,  G06V10/44 ,  G06V10/48 ,  G06V10/75 ,  G06V10/764 ,  G06V10/766 ,  G06V10/776 ,  G06V10/82 ,  G06V10/94 ,  G06V20/56

Abstract: In various examples, systems and methods are disclosed that preserve rich spatial information from an input resolution of a machine learning model to regress on lines in an input image. The machine learning model may be trained to predict, in deployment, distances for each pixel of the input image at an input resolution to a line pixel determined to correspond to a line in the input image. The machine learning model may further be trained to predict angles and label classes of the line. An embedding algorithm may be used to train the machine learning model to predict clusters of line pixels that each correspond to a respective line in the input image. In deployment, the predictions of the machine learning model may be used as an aid for understanding the surrounding environment—e.g., for updating a world model—in a variety of autonomous machine applications.

公开(公告)号：US12236564B2

公开(公告)日：2025-02-25

申请号：US17308663

申请日：2021-05-05

Applicant: NVIDIA Corporation

Inventor： Yining Deng ,  Eric Dujardin ,  Sean Midthun Pieper ,  Minwoo Park

IPC: G06T5/00 ,  G06N3/08 ,  G06T5/50 ,  G06T5/92 ,  H04N5/91 ,  H04N23/741

Abstract: In various examples, apparatuses, systems, and techniques to perform offline image signal processing of source image data to generate target image data. In at least one embodiment, data collection using exposure and calibration setting of an image sensor is performed to generate source image data, which is then processed by using offline image signal processing to generate target data.

公开(公告)号：US20240410705A1

公开(公告)日：2024-12-12

申请号：US18330145

申请日：2023-06-06

Applicant: NVIDIA Corporation

Inventor： Trung Pham ,  Minwoo Park ,  Ha Giang Truong ,  Atchuta Venkata Vijay Chintalapudi ,  Hae-Jong Seo

IPC: G01C21/34 ,  G06N20/00

Abstract: In various examples, path detection using machine learning models for autonomous or semi-autonomous systems and applications is described herein. Systems and methods are disclosed that use one or more machine learning models to determine a geometry associated with a path for a vehicle. To determine the geometry, the machine learning model(s) may process sensor data generated using the vehicle and, based at least on the processing, output points associated with the path. In some examples, the machine learning model(s) outputs a limited number of points, such as between five and twenty points. One or more algorithms, such as one or more Bezier algorithms, may then be used to generate the geometry based at least on the points. As such, in some examples, the geometry may correspond to a Bezier curve that represents the path.

公开(公告)号：US12145617B2

公开(公告)日：2024-11-19

申请号：US17452744

申请日：2021-10-28

Applicant: NVIDIA Corporation

Inventor： Kang Wang ,  Yue Wu ,  Minwoo Park ,  Gang Pan

IPC: B60W60/00 ,  B60W40/06 ,  B60W40/105 ,  G06V20/58

Abstract: In various examples, a 3D surface structure such as the 3D surface structure of a road (3D road surface) may be observed and estimated to generate a 3D point cloud or other representation of the 3D surface structure. Since the representation may be sparse, one or more densification techniques may be applied to densify the representation of the 3D surface structure. For example, the relationship between sparse and dense projection images (e.g., 2D height maps) may be modeled with a Markov random field, and Maximum a Posterior (MAP) inference may be performed using a corresponding joint probability distribution to estimate the most likely dense values given the sparse values. The resulting dense representation of the 3D surface structure may be provided to an autonomous vehicle drive stack to enable safe and comfortable planning and control of the autonomous vehicle.

公开(公告)号：US20240320986A1

公开(公告)日：2024-09-26

申请号：US18734354

申请日：2024-06-05

Applicant: NVIDIA Corporation

Inventor： Mehmet Kocamaz ,  Neeraj Sajjan ,  Sangmin Oh ,  David Nister ,  Junghyun Kwon ,  Minwoo Park

IPC: G06V20/58 ,  G06N3/08 ,  G06V10/20 ,  G06V10/94 ,  G06V20/56 ,  G06V20/64

CPC classification number: G06V20/58 ,  G06N3/08 ,  G06V10/255 ,  G06V10/95 ,  G06V20/588 ,  G06V20/64

Abstract: In various examples, live perception from sensors of an ego-machine may be leveraged to detect objects and assign the objects to bounded regions (e.g., lanes or a roadway) in an environment of the ego-machine in real-time or near real-time. For example, a deep neural network (DNN) may be trained to compute outputs—such as output segmentation masks—that may correspond to a combination of object classification and lane identifiers. The output masks may be post-processed to determine object to lane assignments that assign detected objects to lanes in order to aid an autonomous or semi-autonomous machine in a surrounding environment.

公开(公告)号：US20240232616A9

公开(公告)日：2024-07-11

申请号：US18343291

申请日：2023-06-28

Applicant: NVIDIA Corporation

Inventor： Yilin Yang ,  Bala Siva Sashank Jujjavarapu ,  Pekka Janis ,  Zhaoting Ye ,  Sangmin Oh ,  Minwoo Park ,  Daniel Herrera Castro ,  Tommi Koivisto ,  David Nister

IPC: G06N3/08 ,  B60W30/14 ,  B60W60/00 ,  G06F18/214 ,  G06V10/762 ,  G06V20/56

CPC classification number: G06N3/08 ,  B60W30/14 ,  B60W60/0011 ,  G06F18/2155 ,  G06V10/763 ,  G06V20/56

Abstract: In various examples, a deep neural network (DNN) is trained to accurately predict, in deployment, distances to objects and obstacles using image data alone. The DNN may be trained with ground truth data that is generated and encoded using sensor data from any number of depth predicting sensors, such as, without limitation, RADAR sensors, LIDAR sensors, and/or SONAR sensors. Camera adaptation algorithms may be used in various embodiments to adapt the DNN for use with image data generated by cameras with varying parameters—such as varying fields of view. In some examples, a post-processing safety bounds operation may be executed on the predictions of the DNN to ensure that the predictions fall within a safety-permissible range.

公开(公告)号：US20240176018A1

公开(公告)日：2024-05-30

申请号：US18060444

申请日：2022-11-30

Applicant: NVIDIA Corporation

Inventor： David Weikersdorfer ,  Qian Lin ,  Aman Jhunjhunwala ,  Emilie Lucie Eloïse Wirbel ,  Sangmin Oh ,  Minwoo Park ,  Gyeong Woo Cheon ,  Arthur Henry Rajala ,  Bor-Jeng Chen

IPC: G01S15/931 ,  G01S15/86

CPC classification number: G01S15/931 ,  G01S15/86 ,  G01S2015/938

Abstract: In various examples, techniques for sensor-fusion based object detection and/or free-space detection using ultrasonic sensors are described. Systems may receive sensor data generated using one or more types of sensors of a machine. In some examples, the systems may then process at least a portion of the sensor data to generate input data, where the input data represents one or more locations of one or more objects within an environment. The systems may then input at least a portion of the sensor data and/or at least a portion of the input data into one or more neural networks that are trained to output one or more maps or other output representations associated with the environment. In some examples, the map(s) may include a height, an occupancy, and/or height/occupancy map generated, e.g., from a birds-eye-view perspective. The machine may use these outputs to perform one or more operations

公开(公告)号：US20240127454A1

公开(公告)日：2024-04-18

申请号：US18391276

申请日：2023-12-20

Applicant: NVIDIA Corporation

Inventor： Trung Pham ,  Berta Rodriguez Hervas ,  Minwoo Park ,  David Nister ,  Neda Cvijetic

IPC: G06T7/11 ,  G05B13/02 ,  G06F18/21 ,  G06F18/24 ,  G06N3/04 ,  G06N3/08 ,  G06T3/4046 ,  G06T5/70 ,  G06T11/20 ,  G06V10/26 ,  G06V10/34 ,  G06V10/44 ,  G06V10/82 ,  G06V20/56 ,  G06V30/19 ,  G06V30/262

CPC classification number: G06T7/11 ,  G05B13/027 ,  G06F18/21 ,  G06F18/24 ,  G06N3/04 ,  G06N3/08 ,  G06T3/4046 ,  G06T5/70 ,  G06T11/20 ,  G06V10/267 ,  G06V10/34 ,  G06V10/454 ,  G06V10/82 ,  G06V20/56 ,  G06V30/19173 ,  G06V30/274 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30252 ,  G06T2210/12

Abstract: In various examples, live perception from sensors of a vehicle may be leveraged to detect and classify intersection contention areas in an environment of a vehicle in real-time or near real-time. For example, a deep neural network (DNN) may be trained to compute outputs—such as signed distance functions—that may correspond to locations of boundaries delineating intersection contention areas. The signed distance functions may be decoded and/or post-processed to determine instance segmentation masks representing locations and classifications of intersection areas or regions. The locations of the intersections areas or regions may be generated in image-space and converted to world-space coordinates to aid an autonomous or semi-autonomous vehicle in navigating intersections according to rules of the road, traffic priority considerations, and/or the like.