-
公开(公告)号:US20240404296A1
公开(公告)日:2024-12-05
申请号:US18327643
申请日:2023-06-01
Applicant: NVIDIA Corporation
Inventor: Shagan Sah , Niranjan Avadhanam , Rajath Shetty , Ratin Kumar , Yile Chen
IPC: G06V20/58 , G06T7/20 , G06T7/50 , G06V10/764 , G06V20/59 , G08B13/196
Abstract: In various examples, low power proximity based threat detection using optical flow for vehicle systems and applications are provided. Some embodiments may use a tiered framework that uses sensor fusion techniques to detect and track the movement of a threat candidate, and perform a threat classification and/or intent prediction as the threat candidate approaches approach. Relative depth indications from optical flow, computed using data from image sensors, can be used to initially segment and track a moving object over a sequence of image frames. Additional sensors and processing may be brought online when a moving object becomes close enough to be considered a higher risk threat candidate. A threat response system may generate a risk score based on a predicted intent of a threat candidate, and when the risk score exceeds a certain threshold, then the threat response system may respond accordingly based on the threat classification and/or risk score.
-
公开(公告)号:US11657263B2
公开(公告)日:2023-05-23
申请号:US17005914
申请日:2020-08-28
Applicant: NVIDIA Corporation
Inventor: Nuri Murat Arar , Hairong Jiang , Nishant Puri , Rajath Shetty , Niranjan Avadhanam
IPC: G06K9/62 , G06F18/214 , G06N20/00 , G06V10/94 , G06V20/59 , G06V20/64 , G06V40/16 , G06V40/18 , G06F18/21
CPC classification number: G06F18/214 , G06F18/2193 , G06N20/00 , G06V10/95 , G06V20/597 , G06V20/647 , G06V40/171 , G06V40/193
Abstract: Systems and methods for determining the gaze direction of a subject and projecting this gaze direction onto specific regions of an arbitrary three-dimensional geometry. In an exemplary embodiment, gaze direction may be determined by a regression-based machine learning model. The determined gaze direction is then projected onto a three-dimensional map or set of surfaces that may represent any desired object or system. Maps may represent any three-dimensional layout or geometry, whether actual or virtual. Gaze vectors can thus be used to determine the object of gaze within any environment. Systems can also readily and efficiently adapt for use in different environments by retrieving a different set of surfaces or regions for each environment.
-
13.
公开(公告)号:US11487968B2
公开(公告)日:2022-11-01
申请号:US17004252
申请日:2020-08-27
Applicant: NVIDIA Corporation
Inventor: Nuri Murat Arar , Niranjan Avadhanam , Nishant Puri , Shagan Sah , Rajath Shetty , Sujay Yadawadkar , Pavlo Molchanov
Abstract: Systems and methods for more accurate and robust determination of subject characteristics from an image of the subject. One or more machine learning models receive as input an image of a subject, and output both facial landmarks and associated confidence values. Confidence values represent the degrees to which portions of the subject's face corresponding to those landmarks are occluded, i.e., the amount of uncertainty in the position of each landmark location. These landmark points and their associated confidence values, and/or associated information, may then be input to another set of one or more machine learning models which may output any facial analysis quantity or quantities, such as the subject's gaze direction, head pose, drowsiness state, cognitive load, or distraction state.
-
公开(公告)号:US20210182609A1
公开(公告)日:2021-06-17
申请号:US17005914
申请日:2020-08-28
Applicant: NVIDIA Corporation
Inventor: Nuri Murat Arar , Hairong Jiang , Nishant Puri , Rajath Shetty , Niranjan Avadhanam
Abstract: Systems and methods for determining the gaze direction of a subject and projecting this gaze direction onto specific regions of an arbitrary three-dimensional geometry. In an exemplary embodiment, gaze direction may be determined by a regression-based machine learning model. The determined gaze direction is then projected onto a three-dimensional map or set of surfaces that may represent any desired object or system. Maps may represent any three-dimensional layout or geometry, whether actual or virtual. Gaze vectors can thus be used to determine the object of gaze within any environment. Systems can also readily and efficiently adapt for use in different environments by retrieving a different set of surfaces or regions for each environment.
-
15.
公开(公告)号:US20250022290A1
公开(公告)日:2025-01-16
申请号:US18349853
申请日:2023-07-10
Applicant: NVIDIA Corporation
Inventor: Sakthivel SIVARAMAN , Arjun Guru , Rajath Shetty , Umar Iqbal , Orazio Gallo , Hang Su , Abhishek Badki , Varsha Hedau
Abstract: In various examples, image-based three-dimensional occupant assessment for in-cabin monitoring systems and applications are provided. An evaluation function may determine a 3D representation of an occupant of a machine by evaluating sensor data comprising an image frame from an optical image sensor. The 3D representation may comprise at least one characteristic representative of a size of the occupant, (e.g., a 3D pose and/or 3D shape), which may be used to derive other characteristics such as, but not limited to weight, height, and/or age. A first processing path may generate a representation of one or more features corresponding to at least a portion of the occupant based on optical image data, and a second processing path may determine a depth corresponding to the one or more features based on depth data derived from the optical image data and ground truth depth data corresponding to the interior of the machine.
-
16.
公开(公告)号:US20240095460A1
公开(公告)日:2024-03-21
申请号:US17947491
申请日:2022-09-19
Applicant: NVIDIA Corporation
Inventor: Peng Xu , Mostofa Patwary , Rajath Shetty , Niral Lalit Pathak , Ratin Kumar , Bryan Catanzaro , Mohammad Shoeybi
IPC: G06F40/35
CPC classification number: G06F40/35
Abstract: In various examples, systems and methods that use dialogue systems associated with various machine systems and applications are described. For instance, the systems and methods may receive text data representing speech, such as a question associated with a vehicle or other machine type. The systems and methods then use a retrieval system(s) to retrieve a question/answer pair(s) associated with the text data and/or contextual information associated with the text data. In some examples, the contextual information is associated with a knowledge base associated with or corresponding to the vehicle. The systems and methods then generate a prompt using the text data, the question/answer pair(s), and/or the contextual information. Additionally, the systems and methods determine, using a language model(s) and based at least on the prompt, an output associated with the text data. For instance, the output may include information that answers the question associated with the vehicle.
-
公开(公告)号:US20240087561A1
公开(公告)日:2024-03-14
申请号:US17942950
申请日:2022-09-12
Applicant: NVIDIA Corporation
Inventor: Niral Lalit Pathak , Rajath Shetty , Ratin Kumar
IPC: G10L15/18 , G06F3/01 , G06T7/73 , G10L15/16 , G10L15/183
CPC classification number: G10L15/1822 , G06F3/013 , G06F3/017 , G06T7/73 , G10L15/16 , G10L15/183 , G06T2207/20081
Abstract: In various examples, techniques for using scene-aware context for dialogue systems and applications are described herein. For instance, systems and methods are disclosed that process audio data representing speech in order to determine an intent associated with the speech. Systems and methods are also disclosed that process sensor data representing at least a user in order to determine a point of interest associated with the user. In some examples, the point of interest may include a landmark, a person, and/or any other object within an environment. The systems and methods may then generate a context associated with the point of interest. Additionally, the systems and methods may process the intent and the context using one or more language models. Based on the processing, the language model(s) may output data associated with the speech.
-
公开(公告)号:US20230244941A1
公开(公告)日:2023-08-03
申请号:US18298115
申请日:2023-04-10
Applicant: NVIDIA Corporation
Inventor: Nuri Murat Arar , Hairong Jiang , Nishant Puri , Rajath Shetty , Niranjan Avadhanam
IPC: G06N3/08 , G06N20/00 , G06V10/94 , G06V20/59 , G06V20/64 , G06V40/16 , G06V40/18 , G06F18/214 , G06F18/21 , G06V10/764 , G06V10/774 , G06V10/82
CPC classification number: G06N3/08 , G06F18/214 , G06F18/2193 , G06N20/00 , G06V10/82 , G06V10/95 , G06V10/764 , G06V10/774 , G06V20/597 , G06V20/647 , G06V40/171 , G06V40/193
Abstract: Systems and methods for determining the gaze direction of a subject and projecting this gaze direction onto specific regions of an arbitrary three-dimensional geometry. In an exemplary embodiment, gaze direction may be determined by a regression-based machine learning model. The determined gaze direction is then projected onto a three-dimensional map or set of surfaces that may represent any desired object or system. Maps may represent any three-dimensional layout or geometry, whether actual or virtual. Gaze vectors can thus be used to determine the object of gaze within any environment. Systems can also readily and efficiently adapt for use in different environments by retrieving a different set of surfaces or regions for each environment.
-
19.
公开(公告)号:US20220101047A1
公开(公告)日:2022-03-31
申请号:US17039437
申请日:2020-09-30
Applicant: NVIDIA Corporation
Inventor: Nishant Puri , Sakthivel Sivaraman , Rajath Shetty , Niranjan Avadhanam
Abstract: In various examples, a background of an object may be modified to generate a training image. A segmentation mask may be generated and used to generate an object image that includes image data representing the object. The object image may be integrated into a different background and used for data augmentation in training a neural network. Data augmentation may also be performed using hue adjustment (e.g., of the object image) and/or rendering three-dimensional capture data that corresponds to the object from selected views. Inference scores may be analyzed to select a background for an image to be included in a training dataset. Backgrounds may be selected and training images may be added to a training dataset iteratively during training (e.g., between epochs). Additionally, early or late fusion nay be employed that uses object mask data to improve inferencing performed by a neural network trained using object mask data.
-
-
-
-
-
-
-
-