公开(公告)号：US20220301186A1

公开(公告)日：2022-09-22

申请号：US17678835

申请日：2022-02-23

Applicant: NVIDIA Corporation

Inventor： David Nister ,  Soohwan Kim ,  Yue Wu ,  Minwoo Park ,  Cheng-Chieh Yang

IPC: G06T7/215 ,  G06T7/60 ,  G06V10/422

Abstract: In various examples, an ego-machine may analyze sensor data to identify and track features in the sensor data using. Geometry of the tracked features may be used to analyze motion flow to determine whether the motion flow violates one or more geometrical constraints. As such, tracked features may be identified as dynamic features when the motion flow corresponding to the tracked features violates the one or more static constraints for static features. Tracked features that are determined to be dynamic features may be clustered together according to their location and feature track. Once features have been clustered together, the system may calculate a detection bounding shape for the clustered features. The bounding shape information may then be used by the ego-machine for path planning, control decisions, obstacle avoidance, and/or other operations.

公开(公告)号：US12159417B2

公开(公告)日：2024-12-03

申请号：US17678835

申请日：2022-02-23

Applicant: NVIDIA Corporation

Inventor： David Nister ,  Soohwan Kim ,  Yue Wu ,  Minwoo Park ,  Cheng-Chieh Yang

IPC: G06T7/60 ,  G06T7/215 ,  G06V10/422

Abstract: In various examples, an ego-machine may analyze sensor data to identify and track features in the sensor data using. Geometry of the tracked features may be used to analyze motion flow to determine whether the motion flow violates one or more geometrical constraints. As such, tracked features may be identified as dynamic features when the motion flow corresponding to the tracked features violates the one or more static constraints for static features. Tracked features that are determined to be dynamic features may be clustered together according to their location and feature track. Once features have been clustered together, the system may calculate a detection bounding shape for the clustered features. The bounding shape information may then be used by the ego-machine for path planning, control decisions, obstacle avoidance, and/or other operations.

公开(公告)号：US11579629B2

公开(公告)日：2023-02-14

申请号：US16514404

申请日：2019-07-17

Applicant: NVIDIA Corporation

Inventor： Yue Wu ,  Pekka Janis ,  Xin Tong ,  Cheng-Chieh Yang ,  Minwoo Park ,  David Nister

IPC: G05D1/02 ,  G06K9/62 ,  G06T7/20 ,  G05D1/00 ,  G06V20/10 ,  G06V20/58 ,  G06N3/04

Abstract: In various examples, a sequential deep neural network (DNN) may be trained using ground truth data generated by correlating (e.g., by cross-sensor fusion) sensor data with image data representative of a sequences of images. In deployment, the sequential DNN may leverage the sensor correlation to compute various predictions using image data alone. The predictions may include velocities, in world space, of objects in fields of view of an ego-vehicle, current and future locations of the objects in image space, and/or a time-to-collision (TTC) between the objects and the ego-vehicle. These predictions may be used as part of a perception system for understanding and reacting to a current physical environment of the ego-vehicle.

公开(公告)号：US20220340149A1

公开(公告)日：2022-10-27

申请号：US17726407

申请日：2022-04-21

Applicant: NVIDIA Corporation

Inventor： David Nister ,  Cheng-Chieh Yang ,  Yue Wu

IPC: B60W50/02 ,  B60W60/00

Abstract: In various examples, an end-to-end perception evaluation system for autonomous and semi-autonomous machine applications may be implemented to evaluate how the accuracy or precision of outputs of machine learning models—such as deep neural networks (DNNs)—impact downstream performance of the machine when relied upon. For example, decisions computed by the system using ground truth output types may be compared to decisions computed by the system using the perception outputs. As a result, discrepancies in downstream decision making of the system between the ground truth information and the perception information may be evaluated to either aid in updating or retraining of the machine learning model or aid in generating more accurate or precise ground truth information.

公开(公告)号：US20220250624A1

公开(公告)日：2022-08-11

申请号：US17456835

申请日：2021-11-29

Applicant: NVIDIA Corporation

Inventor： Minwoo Park ,  Yue Wu ,  Cheng-Chieh Yang

IPC: B60W40/02 ,  G06T7/593 ,  G06T7/30 ,  G06T7/80 ,  H04N13/296 ,  H04N13/271 ,  G06T15/10 ,  G06V20/58 ,  B60W60/00

Abstract: In various examples, systems and methods are disclosed that detect hazards on a roadway by identifying discontinuities between pixels on a depth map. For example, two synchronized stereo cameras mounted on an ego-machine may generate images that may be used extract depth or disparity information. Because a hazard's height may cause an occlusion of the driving surface behind the hazard from a perspective of a camera(s), a discontinuity in disparity values may indicate the presence of a hazard. For example, the system may analyze pairs of pixels on the depth map and, when the system determines that a disparity between a pair of pixels satisfies a disparity threshold, the system may identify the pixel nearest the ego-machine as a hazard pixel.

公开(公告)号：US20210183093A1

公开(公告)日：2021-06-17

申请号：US17103680

申请日：2020-11-24

Applicant: NVIDIA Corporation

Inventor： Minwoo Park ,  Yue Wu ,  Michael Grabner ,  Cheng-Chieh Yang

IPC: G06T7/60 ,  G06T7/579 ,  G06K9/00

Abstract: In various examples, surface profile estimation and bump detection may be performed based on a three-dimensional (3D) point cloud. The 3D point cloud may be filtered in view of a portion of an environment including drivable free-space, and within a threshold height to factor out other objects or obstacles other than a driving surface and protuberances thereon. The 3D point cloud may be analyzed—e.g., using a sliding window of bounding shapes along a longitudinal or other heading direction—to determine one-dimensional (1D) signal profiles corresponding to heights along the driving surface. The profile itself may be used by a vehicle—e.g., an autonomous or semi-autonomous vehicle—to help in navigating the environment, and/or the profile may be used to detect bumps, humps, and/or other protuberances along the driving surface, in addition to a location, orientation, and geometry thereof

公开(公告)号：US12269488B2

公开(公告)日：2025-04-08

申请号：US17726407

申请日：2022-04-21

Applicant: NVIDIA Corporation

Inventor： David Nister ,  Cheng-Chieh Yang ,  Yue Wu

IPC: B60W50/02 ,  B60W60/00

Abstract: In various examples, an end-to-end perception evaluation system for autonomous and semi-autonomous machine applications may be implemented to evaluate how the accuracy or precision of outputs of machine learning models—such as deep neural networks (DNNs)—impact downstream performance of the machine when relied upon. For example, decisions computed by the system using ground truth output types may be compared to decisions computed by the system using the perception outputs. As a result, discrepancies in downstream decision making of the system between the ground truth information and the perception information may be evaluated to either aid in updating or retraining of the machine learning model or aid in generating more accurate or precise ground truth information.

公开(公告)号：US20250029264A1

公开(公告)日：2025-01-23

申请号：US18905939

申请日：2024-10-03

Applicant: NVIDIA Corporation

Inventor： David Nister ,  Soohwan Kim ,  Yue Wu ,  Minwoo Park ,  Cheng-Chieh Yang

IPC: G06T7/215 ,  G06T7/60 ,  G06V10/422

Abstract: In various examples, an ego-machine may analyze sensor data to identify and track features in the sensor data using. Geometry of the tracked features may be used to analyze motion flow to determine whether the motion flow violates one or more geometrical constraints. As such, tracked features may be identified as dynamic features when the motion flow corresponding to the tracked features violates the one or more static constraints for static features. Tracked features that are determined to be dynamic features may be clustered together according to their location and feature track. Once features have been clustered together, the system may calculate a detection bounding shape for the clustered features. The bounding shape information may then be used by the ego-machine for path planning, control decisions, obstacle avoidance, and/or other operations.

公开(公告)号：US20250022175A1

公开(公告)日：2025-01-16

申请号：US18349779

申请日：2023-07-10

Applicant: NVIDIA Corporation

Inventor： Yue Wu ,  Cheng-Chieh Yang ,  Kang Wang ,  Ayon Sen ,  Hsin Miao

IPC: G06T7/80 ,  G06T7/73 ,  H04N13/246

Abstract: In various examples, sensor calibration for autonomous or semi-autonomous systems and applications is described herein. Systems and methods are disclosed that calibrate image sensors, such as cameras, using images captured by the image sensors at different time instances. For instance, a first image sensor may generate first image data representing at least two images and a second image sensor may generate second image data representing at least one image. One or more feature points may then be tracked between the images represented by the first image data and the image represented by the second image data. Additionally, the feature point(s), timestamps associated with the images, poses associated with image sensors (e.g., poses of a vehicle), and/or other information may be used to determine one or more values of one or more parameters that calibrate the first image sensor with the second image sensor.

公开(公告)号：US20240312187A1

公开(公告)日：2024-09-19

申请号：US18184071

申请日：2023-03-15

Applicant: NVIDIA Corporation

Inventor： Yue Wu ,  Cheng-Chieh Yang ,  Xin Tong ,  Minwoo Park

IPC: G06V10/771 ,  G06V10/77

CPC classification number: G06V10/771 ,  G06V10/7715

Abstract: In various examples, feature tracking for autonomous or semi-autonomous systems and applications is described herein. Systems and methods are disclosed that merge, using one or more processes, features detected using a feature tracker(s) and features detected using a feature detector(s) in order to track features between images. In some examples, the number of merged features and/or the locations of the merged features within the images are limited. This way, the systems and methods are able to identify merged features that are of greater importance for tracking while refraining from tracking merged features that are of less importance. For example, if the systems and methods are being used to identify features for autonomous driving, a greater number of merged features that are associated with objects located proximate to the driving surface may be tracked as compared to merged features that are associated with the sky.