公开(公告)号：US20250131212A1

公开(公告)日：2025-04-24

申请号：US18919630

申请日：2024-10-18

Applicant: SRI International

Inventor： Pengfei Yu ,  Yi Yao ,  Karan Sikka ,  Michael A. Cogswell ,  Ajay Divakaran

IPC: G06F40/56

Abstract: In an example, a method for generating responses by a Machine Learning (ML) system includes processing, by a first language model, a natural language instruction to generate an instruction representation based on a meaning of the natural language instruction; translating, by a translation module comprising an interface between the first language model and a second language model, the instruction representation into data indicating an intent of the natural language instruction, wherein the second language model is trained with domain specific knowledge; providing, by the translation module, the natural language instruction and the data indicating the intent of the natural language instruction to the second language model; and generating, by the second language model, a response based on the natural language instruction and the data indicating the intent of the natural language instruction.

公开(公告)号：US20240005654A1

公开(公告)日：2024-01-04

申请号：US17656391

申请日：2022-03-24

Applicant: SRI International

Inventor： Arijit Ray ,  Michael A. Cogswell ,  Ajay Divakaran ,  Yi Yao ,  Giedrius T. Burachas ,  Kamran Alipour

IPC: G06V10/98 ,  G06T11/00 ,  G06V10/776 ,  G06V10/77

CPC classification number: G06V10/98 ,  G06T11/001 ,  G06V10/776 ,  G06V10/7715

Abstract: A computing system comprising a memory configured to store an artificial intelligence (AI) model and an image, and a computation engine executing one or more processors may be configured to perform the techniques for error-based explanations for AI behavior. The computation engine may execute the AI model to analyze the image to output a result. The AI model may, when analyzing the image to output the result, process, based on data indicative of the result, the image to assign an error score to each image feature extracted from the image, and obtain, based on the error scores, an error map. The AI model may next update, based on the error map and to obtain a first updated image, the image to visually indicate the error score assigned to each of the image features, and output one or more of the error scores, the error map, and the first updated image.

公开(公告)号：US20210192972A1

公开(公告)日：2021-06-24

申请号：US17129541

申请日：2020-12-21

Applicant: SRI International

Inventor： Girish Acharya ,  Louise Yarnall ,  Anirban Roy ,  Michael Wessel ,  Yi Yao ,  John J. Byrnes ,  Dayne Freitag ,  Zachary Weiler ,  Paul Kalmar

IPC: G09B19/00 ,  G06K9/72 ,  G10L15/18 ,  G06K9/00 ,  G10L25/57 ,  G06K9/62 ,  G09B5/06 ,  G06N20/00

Abstract: This disclosure describes machine learning techniques for capturing human knowledge for performing a task. In one example, a video device obtains video data of a first user performing the task and one or more sensors generate sensor data during performance of the task. An audio device obtains audio data describing performance of the task. A computation engine applies a machine learning system to correlate the video data to the audio data and sensor data to identify portions of the video, sensor, and audio data that depict a same step of a plurality of steps for performing the task. The machine learning system further processes the correlated data to update a domain model defining performance of the task. A training unit applies the domain model to generate training information for performing the task. An output device outputs the training information for use in training a second user to perform the task.

公开(公告)号：US10909401B2

公开(公告)日：2021-02-02

申请号：US16422649

申请日：2019-05-24

Applicant: SRI International

Inventor： Giedrius Burachas ,  Arijit Ray ,  Yi Yao

IPC: G06K9/20 ,  G06N20/00 ,  G06F16/53 ,  G06T11/20 ,  G06T11/60 ,  G06T7/11

Abstract: In general, the disclosure describes various aspects of techniques for attention-based explanations for artificial intelligence behavior. A device comprising a memory and a computation engine executing a processor may be configured to perform the techniques. The memory may store the artificial intelligence model and the image. The computation engine may receive a query regarding the image, and execute the artificial intelligence model to analyze the image in order to output the result to the query. The artificial intelligence model may, when analyzing the image to output the result, segment the image into hierarchically arranged semantic areas in which objects in the image are segmented into parts, determine, based on the query, an attention mask for the areas, update, based on the attention mask, the image to visually identify which of the areas formed a basis for the result, and output the updated image.

公开(公告)号：US20250110989A1

公开(公告)日：2025-04-03

申请号：US18895080

申请日：2024-09-24

Applicant: SRI International

Inventor： Ajay Divakaran ,  Yi Yao ,  Julia Kruk ,  Jesse Hostetler ,  Jihua Huang

IPC: G06F16/901

Abstract: In general, various aspects of the techniques are directed to causal analysis using large scale time series data. A computing system may convert large scale time series data to first time period records and second time period records according to a multi-scale time resolution. The computing system may implement a hierarchical machine learning model to generate embeddings that capture temporal characteristics of features of the large scale time series data. The computing system may generate a graph data structure indicating cause and effect correlations between features of the large scale time series data based on temporal dynamics captured in the cause and second time period records and/or the embeddings.

公开(公告)号：US20240403728A1

公开(公告)日：2024-12-05

申请号：US18614388

申请日：2024-03-22

Applicant: SRI International

Inventor： Yunye Gong ,  Yi Yao ,  Xiao Lin ,  Ajay Divakaran

IPC: G06N20/20 ,  G06N7/01

Abstract: In general, techniques are described that address the limitations of existing conformal prediction methods for cascaded models. In an example, a method includes receiving a first validation data set for validating performance of an upstream model of the two or more cascaded models and receiving a second validation data set for validating performance of a downstream model of the two or more cascaded models wherein the second validation data set is different than the first validation set; estimating system-level errors caused by predictions of the upstream model based on the first validation data set; estimating system-level errors caused by predictions of the downstream model based on the second validation data set; and generating a prediction confidence interval that indicates a confidence for the system based on the system-level errors caused by predictions of the upstream model and based on the system-level errors caused by predictions of the downstream model.

公开(公告)号：US20240403649A1

公开(公告)日：2024-12-05

申请号：US18520800

申请日：2023-11-28

Applicant: SRI International

Inventor： Han-Pang Chiu ,  Yi Yao ,  Zachary Seymour ,  Alex Krasner ,  Bradley J. Clymer ,  Michael A. Cogswell ,  Cecile Eliane Jeannine Mackay ,  Alex C. Tozzo ,  Tixiao Shan ,  Philip Miller ,  Chuanyong Gan ,  Glenn A. Murray ,  Richard Louis Ferranti ,  Uma Rajendran ,  Supun Samarasekera ,  Rakesh Kumar ,  James Smith

IPC: G06N3/0895

Abstract: In an example, a system includes processing circuitry in communication with storage media. The processing circuitry is configured to execute a machine learning system including at least a first module, a second module and a third module. The machine learning system is configured to train one or more machine learning models. The first module is configured to generate augmented input data based on the streaming input data. The second module includes a machine learning model configured to perform a specific task based at least in part on the augmented input data. The third module configured to adapt a network architecture of the one or more machine learning models based on changes in the streaming input data.

公开(公告)号：US12118773B2

公开(公告)日：2024-10-15

申请号：US17129541

申请日：2020-12-21

Applicant: SRI International

Inventor： Girish Acharya ,  Louise Yarnall ,  Anirban Roy ,  Michael Wessel ,  Yi Yao ,  John J. Byrnes ,  Dayne Freitag ,  Zachary Weiler ,  Paul Kalmar

IPC: G06V10/82 ,  G06F18/22 ,  G06N20/00 ,  G06V20/20 ,  G06V20/40 ,  G06V30/19 ,  G06V30/262 ,  G06V40/10 ,  G06V40/20 ,  G09B5/06 ,  G09B19/00 ,  G10L15/18 ,  G10L25/57

CPC classification number: G06V10/82 ,  G06F18/22 ,  G06N20/00 ,  G06V20/20 ,  G06V20/41 ,  G06V30/19173 ,  G06V30/274 ,  G06V40/10 ,  G06V40/113 ,  G06V40/28 ,  G09B5/065 ,  G09B19/003 ,  G10L15/1815 ,  G10L25/57

Abstract: This disclosure describes machine learning techniques for capturing human knowledge for performing a task. In one example, a video device obtains video data of a first user performing the task and one or more sensors generate sensor data during performance of the task. An audio device obtains audio data describing performance of the task. A computation engine applies a machine learning system to correlate the video data to the audio data and sensor data to identify portions of the video, sensor, and audio data that depict a same step of a plurality of steps for performing the task. The machine learning system further processes the correlated data to update a domain model defining performance of the task. A training unit applies the domain model to generate training information for performing the task. An output device outputs the training information for use in training a second user to perform the task.

公开(公告)号：US11790213B2

公开(公告)日：2023-10-17

申请号：US16439508

申请日：2019-06-12

Applicant: SRI International

Inventor： Yi Yao ,  Ajay Divakaran ,  Pallabi Ghosh

IPC: G06N3/045 ,  G06N3/08

CPC classification number: G06N3/045 ,  G06N3/08

Abstract: Techniques are disclosed for identifying multimodal subevents within an event having spatially-related and temporally-related features. In one example, a system receives a Spatio-Temporal Graph (STG) comprising (1) a plurality of nodes, each node having a feature descriptor that describes a feature present in the event, (2) a plurality of spatial edges, each spatial edge describing a spatial relationship between two of the plurality of nodes, and (3) a plurality of temporal edges, each temporal edge describing a temporal relationship between two of the plurality of nodes. Furthermore, the STG comprises at least one of: (1) variable-length descriptors for the feature descriptors or (2) temporal edges that span multiple time steps for the event. A machine learning system processes the STG to identify the multimodal subevents for the event. In some examples, the machine learning system comprises stacked Spatio-Temporal Graph Convolutional Networks (STGCNs), each comprising a plurality of STGCN layers.

公开(公告)号：US20210390400A1

公开(公告)日：2021-12-16

申请号：US17304163

申请日：2021-06-15

Applicant: SRI International

Inventor： Yi Yao ,  Ajay Divakaran ,  Hammad A. Ayyubi

IPC: G06N3/08 ,  G06F17/13 ,  G06K9/62

Abstract: Techniques are described for neural networks based on Progressive Neural ODEs (PODEs). In an example, a method to progressively train a neural ordinary differential equation (NODE) model comprises processing, by a machine learning system executed by a computing system, first training data, the first training data having a first complexity, to perform training of a first layer for the NODE model; and after performing the first training, processing second training data, the second training data having a second complexity that is higher than the first complexity, to perform training of a second layer for the NODE model.