公开(公告)号：US20250067581A1

公开(公告)日：2025-02-27

申请号：US18456070

申请日：2023-08-25

Applicant: NVIDIA Corporation

Inventor： Dae Jin Kim ,  Rajath Shetty

IPC: G01D21/00 ,  G06T7/80

Abstract: In various examples, interior sensor calibration for autonomous systems and applications is described herein. Systems and methods are disclosed that may recalibrate sensors of a vehicle, such as sensors located within the interior of the vehicle, using one or more techniques. For instance, if a sensor is attached to a component within the interior of the vehicle, an additional sensor associated with the component may output data indicating the location and/or orientation of the component within the vehicle. The indicated location and/or orientation of the component may then be used to recalibrate the sensor with respect to a reference coordinate system of the vehicle. For a second example, the sensor may output data representing at least a feature located within the interior of the vehicle. The sensor may then again be recalibrated based at least on a portion of the data that represents the feature.

公开(公告)号：US20230078171A1

公开(公告)日：2023-03-16

申请号：US18051296

申请日：2022-10-31

Applicant: NVIDIA Corporation

Inventor： Nuri Murat Arar ,  Niranjan Avadhanam ,  Nishant Puri ,  Shagan Sah ,  Rajath Shetty ,  Sujay Yadawadkar ,  Pavlo Molchanov

IPC: G06K9/62 ,  G06N20/00 ,  G06V10/94 ,  G06V20/59 ,  G06V20/64 ,  G06V40/16 ,  G06V40/18

Abstract: Systems and methods for more accurate and robust determination of subject characteristics from an image of the subject. One or more machine learning models receive as input an image of a subject, and output both facial landmarks and associated confidence values. Confidence values represent the degrees to which portions of the subject's face corresponding to those landmarks are occluded, i.e., the amount of uncertainty in the position of each landmark location. These landmark points and their associated confidence values, and/or associated information, may then be input to another set of one or more machine learning models which may output any facial analysis quantity or quantities, such as the subject's gaze direction, head pose, drowsiness state, cognitive load, or distraction state.

公开(公告)号：US11934955B2

公开(公告)日：2024-03-19

申请号：US18051296

申请日：2022-10-31

Applicant: NVIDIA Corporation

Inventor： Nuri Murat Arar ,  Niranjan Avadhanam ,  Nishant Puri ,  Shagan Sah ,  Rajath Shetty ,  Sujay Yadawadkar ,  Pavlo Molchanov

IPC: G06N3/08 ,  G06F18/21 ,  G06F18/214 ,  G06N20/00 ,  G06V10/764 ,  G06V10/774 ,  G06V10/82 ,  G06V10/94 ,  G06V20/59 ,  G06V20/64 ,  G06V40/16 ,  G06V40/18

CPC classification number: G06N3/08 ,  G06F18/214 ,  G06F18/2193 ,  G06N20/00 ,  G06V10/764 ,  G06V10/774 ,  G06V10/82 ,  G06V10/95 ,  G06V20/597 ,  G06V20/647 ,  G06V40/171 ,  G06V40/193

Abstract: Systems and methods for more accurate and robust determination of subject characteristics from an image of the subject. One or more machine learning models receive as input an image of a subject, and output both facial landmarks and associated confidence values. Confidence values represent the degrees to which portions of the subject's face corresponding to those landmarks are occluded, i.e., the amount of uncertainty in the position of each landmark location. These landmark points and their associated confidence values, and/or associated information, may then be input to another set of one or more machine learning models which may output any facial analysis quantity or quantities, such as the subject's gaze direction, head pose, drowsiness state, cognitive load, or distraction state.

公开(公告)号：US11688074B2

公开(公告)日：2023-06-27

申请号：US17039437

申请日：2020-09-30

Applicant: NVIDIA Corporation

Inventor： Nishant Puri ,  Sakthivel Sivaraman ,  Rajath Shetty ,  Niranjan Avadhanam

IPC: G06T7/194 ,  G06T5/30 ,  G06T5/00 ,  G06N3/08 ,  G06V40/10 ,  G06F18/214 ,  G06F18/24

CPC classification number: G06T7/194 ,  G06F18/214 ,  G06F18/24 ,  G06N3/08 ,  G06T5/002 ,  G06T5/30 ,  G06V40/11 ,  G06V40/113 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20132 ,  G06T2207/20221 ,  G06T2207/30196

Abstract: In various examples, a background of an object may be modified to generate a training image. A segmentation mask may be generated and used to generate an object image that includes image data representing the object. The object image may be integrated into a different background and used for data augmentation in training a neural network. Data augmentation may also be performed using hue adjustment (e.g., of the object image) and/or rendering three-dimensional capture data that corresponds to the object from selected views. Inference scores may be analyzed to select a background for an image to be included in a training dataset. Backgrounds may be selected and training images may be added to a training dataset iteratively during training (e.g., between epochs). Additionally, early or late fusion nay be employed that uses object mask data to improve inferencing performed by a neural network trained using object mask data.

公开(公告)号：US20240265254A1

公开(公告)日：2024-08-08

申请号：US18605628

申请日：2024-03-14

Applicant: NVIDIA Corporation

Inventor： Nuri Murat Arar ,  Niranjan Avadhanam ,  Nishant Puri ,  Shagan Sah ,  Rajath Shetty ,  Sujay Yadawadkar ,  Pavlo Molchanov

IPC: G06N3/08 ,  G06F18/21 ,  G06F18/214 ,  G06N20/00 ,  G06V10/764 ,  G06V10/774 ,  G06V10/82 ,  G06V10/94 ,  G06V20/59 ,  G06V20/64 ,  G06V40/16 ,  G06V40/18

CPC classification number: G06N3/08 ,  G06F18/214 ,  G06F18/2193 ,  G06N20/00 ,  G06V10/764 ,  G06V10/774 ,  G06V10/82 ,  G06V10/95 ,  G06V20/597 ,  G06V20/647 ,  G06V40/171 ,  G06V40/193

Abstract: Systems and methods for more accurate and robust determination of subject characteristics from an image of the subject. One or more machine learning models receive as input an image of a subject, and output both facial landmarks and associated confidence values. Confidence values represent the degrees to which portions of the subject's face corresponding to those landmarks are occluded, i.e., the amount of uncertainty in the position of each landmark location. These landmark points and their associated confidence values, and/or associated information, may then be input to another set of one or more machine learning models which may output any facial analysis quantity or quantities, such as the subject's gaze direction, head pose, drowsiness state, cognitive load, or distraction state.

公开(公告)号：US20240022601A1

公开(公告)日：2024-01-18

申请号：US17863140

申请日：2022-07-12

Applicant: NVIDIA Corporation

Inventor： Manoj Kumar Yennapureddy ,  Shagan Sah ,  Rajath Shetty

IPC: H04L9/40 ,  G06T7/50

CPC classification number: H04L63/1466 ,  H04L63/1416 ,  G06T7/50 ,  G06T2207/10028 ,  G06T2207/20084 ,  G06T2207/20076

Abstract: In various examples, techniques are described for detecting whether spoofing attacks are occurring using multiple sensors. Systems and methods are disclosed that include at least a first sensor having a first pose to capture a first perspective view of a user and a second sensor having a second pose to capture a second perspective view of the user. The first sensor and/or the second sensor may include an image sensor, a depth sensor, and/or the like. The systems and methods include a neural network that is configured to analyze first sensor data generated by the first sensor and second sensor data generated by the second sensor to determine whether a spoofing attack is occurring. The systems and methods may also perform one or more processes, such as facial recognition, based on whether the spoofing attack is occurring.

公开(公告)号：US11704814B2

公开(公告)日：2023-07-18

申请号：US17319891

申请日：2021-05-13

Applicant: NVIDIA Corporation

Inventor： Nuri Murat Arar ,  Niranjan Avadhanam ,  Hairong Jiang ,  Nishant Puri ,  Rajath Shetty ,  Shagan Sah

IPC: G06T7/20 ,  G06N3/08 ,  G06K9/00 ,  G06V20/59 ,  G06V40/18 ,  G06F18/211 ,  G06F18/21

CPC classification number: G06T7/20 ,  G06F18/211 ,  G06F18/2178 ,  G06N3/08 ,  G06V20/59 ,  G06V40/18

Abstract: In various examples, an adaptive eye tracking machine learning model engine (“adaptive-model engine”) for an eye tracking system is described. The adaptive-model engine may include an eye tracking or gaze tracking development pipeline (“adaptive-model training pipeline”) that supports collecting data, training, optimizing, and deploying an adaptive eye tracking model that is a customized eye tracking model based on a set of features of an identified deployment environment. The adaptive-model engine supports ensembling the adaptive eye tracking model that may be trained on gaze vector estimation in surround environments and ensemble based on a plurality of eye tracking variant models and a plurality of facial landmark neural network metrics.

公开(公告)号：US20220366568A1

公开(公告)日：2022-11-17

申请号：US17319891

申请日：2021-05-13

Applicant: NVIDIA Corporation

Inventor： Nuri Murat Arar ,  Niranjan Avadhanam ,  Hairong Jiang ,  Nishant Puri ,  Rajath Shetty ,  Shagan Sah

IPC: G06T7/20 ,  G06K9/00 ,  G06K9/62 ,  G06N3/08

Abstract: In various examples, an adaptive eye tracking machine learning model engine (“adaptive-model engine”) for an eye tracking system is described. The adaptive-model engine may include an eye tracking or gaze tracking development pipeline (“adaptive-model training pipeline”) that supports collecting data, training, optimizing, and deploying an adaptive eye tracking model that is a customized eye tracking model based on a set of features of an identified deployment environment. The adaptive-model engine supports ensembling the adaptive eye tracking model that may be trained on gaze vector estimation in surround environments and ensemble based on a plurality of eye tracking variant models and a plurality of facial landmark neural network metrics.

公开(公告)号：US20210182625A1

公开(公告)日：2021-06-17

申请号：US17004252

申请日：2020-08-27

Applicant: NVIDIA Corporation

Inventor： Nuri Murat Arar ,  Niranjan Avadhanam ,  Nishant Puri ,  Shagan Sah ,  Rajath Shetty ,  Sujay Yadawadkar ,  Pavlo Molchanov

IPC: G06K9/62 ,  G06K9/00

Abstract: Systems and methods for more accurate and robust determination of subject characteristics from an image of the subject. One or more machine learning models receive as input an image of a subject, and output both facial landmarks and associated confidence values. Confidence values represent the degrees to which portions of the subject's face corresponding to those landmarks are occluded, i.e., the amount of uncertainty in the position of each landmark location. These landmark points and their associated confidence values, and/or associated information, may then be input to another set of one or more machine learning models which may output any facial analysis quantity or quantities, such as the subject's gaze direction, head pose, drowsiness state, cognitive load, or distraction state.

公开(公告)号：US20250078315A1

公开(公告)日：2025-03-06

申请号：US18240802

申请日：2023-08-31

Applicant: NVIDIA Corporation

Inventor： Dae Jin Kim ,  Rajath Shetty

IPC: G06T7/80 ,  G01S7/497 ,  G06T7/70 ,  G06V20/56

Abstract: In various examples, interior sensor calibration using exterior features for autonomous systems and applications is described herein. Systems and methods are disclosed that use one or more features that are located exterior to a vehicle, such as one or more tags located in the environment surrounding the vehicle, to determine one or more values for one or more calibration parameters that calibrate an interior sensor of the vehicle with respect to a reference coordinate system of the vehicle. For instance, such as when the vehicle is located at a calibration station, the sensor may generate sensor data representing a sensor representation, where at least a portion of the sensor representation is associated with a feature that is visible through a transparent component (e.g., a window) of the vehicle. The sensor data may then be used to calibrate the sensor with respect to the coordinate system.