公开(公告)号：US20240004677A1

公开(公告)日：2024-01-04

申请号：US18466267

申请日：2023-09-13

Applicant: Google LLC

Inventor： Srinivas Kumar Sunkara ,  Xiaoxue Zang ,  Ying Xu ,  Lijuan Liu ,  Nevan Holt Wichers ,  Gabriel Overholt Schubiner ,  Jindong Chen ,  Abhinav Kumar Rastogi ,  Blaise Aguera-Arcas ,  Zecheng He

IPC: G06F9/451 ,  G06N20/00 ,  G06F18/214 ,  G06F18/2135 ,  G06N3/045

CPC classification number: G06F9/451 ,  G06N20/00 ,  G06N3/045 ,  G06F18/21355 ,  G06F18/214

Abstract: Generally, the present disclosure is directed to user interface understanding. More particularly, the present disclosure relates to training and utilization of machine-learned models for user interface prediction and/or generation. A machine-learned interface prediction model can be pre-trained using a variety of pre-training tasks for eventual downstream task training and utilization (e.g., interface prediction, interface generation, etc.).

公开(公告)号：US11789753B2

公开(公告)日：2023-10-17

申请号：US17335596

申请日：2021-06-01

Applicant: Google LLC

Inventor： Srinivas Kumar Sunkara ,  Xiaoxue Zang ,  Ying Xu ,  Lijuan Liu ,  Nevan Holt Wichers ,  Gabriel Overholt Schubiner ,  Jindong Chen ,  Abhinav Kumar Rastogi ,  Blaise Aguera-Arcas ,  Zecheng He

IPC: G06F9/44 ,  G06F9/451 ,  G06N20/00 ,  G06F18/214 ,  G06F18/2135 ,  G06N3/045

CPC classification number: G06F9/451 ,  G06F18/214 ,  G06F18/21355 ,  G06N3/045 ,  G06N20/00

Abstract: Generally, the present disclosure is directed to user interface understanding. More particularly, the present disclosure relates to training and utilization of machine-learned models for user interface prediction and/or generation. A machine-learned interface prediction model can be pre-trained using a variety of pre-training tasks for eventual downstream task training and utilization (e.g., interface prediction, interface generation, etc.).

公开(公告)号：US20240274132A1

公开(公告)日：2024-08-15

申请号：US18642010

申请日：2024-04-22

Applicant: GOOGLE LLC

Inventor： Joseph Lange ,  Abhanshu Sharma ,  Adam Coimbra ,  Gökhan Bakir ,  Gabriel Taubman ,  IIya Firman ,  Jindong Chen ,  James Stout ,  Marcin Nowak-Przygodzki ,  Reed Enger ,  Thomas Weedon Hume ,  Vishwath Mohan ,  Jacek Szmigiel ,  Yunfan Jin ,  Kyle Pedersen ,  Gilles Baechler

IPC: G10L15/22 ,  G06F3/16 ,  G06F40/247 ,  G06F40/30 ,  G10L15/18

CPC classification number: G10L15/22 ,  G06F3/167 ,  G06F40/247 ,  G06F40/30 ,  G10L15/1815 ,  G10L15/1822 ,  G10L2015/223 ,  G10L2015/228

Abstract: Implementations set forth herein relate to an automated assistant that can interact with applications that may not have been pre-configured for interfacing with the automated assistant. The automated assistant can identify content of an application interface of the application to determine synonymous terms that a user may speak when commanding the automated assistant to perform certain tasks. Speech processing operations employed by the automated assistant can be biased towards these synonymous terms when the user is accessing an application interface of the application. In some implementations, the synonymous terms can be identified in a responsive language of the automated assistant when the content of the application interface is being rendered in a different language. This can allow the automated assistant to operate as an interface between the user and certain applications that may not be rendering content in a native language of the user.

公开(公告)号：US20240169186A1

公开(公告)日：2024-05-23

申请号：US18550203

申请日：2021-06-02

Applicant: Google LLC

Inventor： Xiaoxue Zang ,  Ying Xu ,  Srinivas Kumar Sunkara ,  Abhinav Kumar Rastogi ,  Jindong Chen ,  Blaise Aguera-Arcas ,  Chongyang Bai

IPC: G06N3/0455 ,  G06N3/084

CPC classification number: G06N3/0455 ,  G06N3/084

Abstract: Generally, the present disclosure is directed to user interface understanding. More particularly, the present disclosure relates to training and utilization of machine-learned models for user interface prediction and/or generation. A machine-learned interface Nprediction model can be pre-trained using a variety of pre-training tasks for eventual downstream task training and utilization (e.g., interface prediction, interface generation, etc.).

公开(公告)号：US20250117232A1

公开(公告)日：2025-04-10

申请号：US18988564

申请日：2024-12-19

Applicant: Google LLC

Inventor： Srinivas Kumar Sunkara ,  Xiaoxue Zang ,  Ying Xu ,  Lijuan Liu ,  Nevan Holt Wichers ,  Gabriel Overholt Schubiner ,  Jindong Chen ,  Abhinav Kumar Rastogi ,  Blaise Aguera-Arcas ,  Zecheng He

IPC: G06F9/451

Abstract: Generally, the present disclosure is directed to user interface understanding. More particularly, the present disclosure relates to training and utilization of machine-learned models for user interface prediction and/or generation. A machine-learned interface prediction model can be pre-trained using a variety of pre-training tasks for eventual downstream task training and utilization (e.g., interface prediction, interface generation, etc.).

公开(公告)号：US12197930B2

公开(公告)日：2025-01-14

申请号：US18466267

申请日：2023-09-13

Applicant: Google LLC

Inventor： Srinivas Kumar Sunkara ,  Xiaoxue Zang ,  Ying Xu ,  Lijuan Liu ,  Nevan Holt Wichers ,  Gabriel Overholt Schubiner ,  Jindong Chen ,  Abhinav Kumar Rastogi ,  Blaise Aguera-Arcas ,  Zecheng He

IPC: G06F9/44 ,  G06F9/451 ,  G06F18/2135 ,  G06F18/214 ,  G06N3/045 ,  G06N20/00

Abstract: Generally, the present disclosure is directed to user interface understanding. More particularly, the present disclosure relates to training and utilization of machine-learned models for user interface prediction and/or generation. A machine-learned interface prediction model can be pre-trained using a variety of pre-training tasks for eventual downstream task training and utilization (e.g., interface prediction, interface generation, etc.).

公开(公告)号：US11967321B2

公开(公告)日：2024-04-23

申请号：US17538641

申请日：2021-11-30

Applicant: GOOGLE LLC

Inventor： Joseph Lange ,  Abhanshu Sharma ,  Adam Coimbra ,  Gökhan Bakir ,  Gabriel Taubman ,  Ilya Firman ,  Jindong Chen ,  James Stout ,  Marcin Nowak-Przygodzki ,  Reed Enger ,  Thomas Weedon Hume ,  Vishwath Mohan ,  Jacek Szmigiel ,  Yunfan Jin ,  Kyle Pedersen ,  Gilles Baechler

IPC: G10L15/22 ,  G06F3/16 ,  G06F40/247 ,  G06F40/30 ,  G10L15/18

CPC classification number: G10L15/22 ,  G06F3/167 ,  G06F40/247 ,  G06F40/30 ,  G10L15/1815 ,  G10L15/1822 ,  G10L2015/223 ,  G10L2015/228

Abstract: Implementations set forth herein relate to an automated assistant that can interact with applications that may not have been pre-configured for interfacing with the automated assistant. The automated assistant can identify content of an application interface of the application to determine synonymous terms that a user may speak when commanding the automated assistant to perform certain tasks. Speech processing operations employed by the automated assistant can be biased towards these synonymous terms when the user is accessing an application interface of the application. In some implementations, the synonymous terms can be identified in a responsive language of the automated assistant when the content of the application interface is being rendered in a different language. This can allow the automated assistant to operate as an interface between the user and certain applications that may not be rendering content in a native language of the user.

公开(公告)号：US20220382565A1

公开(公告)日：2022-12-01

申请号：US17335596

申请日：2021-06-01

Applicant: Google LLC

Inventor： Srinivas Kumar Sunkara ,  Xiaoxue Zang ,  Ying Xu ,  Lijuan Liu ,  Nevan Holt Wichers ,  Gabriel Overholt Schubiner ,  Jindong Chen ,  Abhinav Kumar Rastogi ,  Blaise Aguera-Arcas ,  Zecheng He

IPC: G06F9/451 ,  G06N20/00 ,  G06N3/04 ,  G06K9/62

Abstract: Generally, the present disclosure is directed to user interface understanding. More particularly, the present disclosure relates to training and utilization of machine-learned models for user interface prediction and/or generation. A machine-learned interface prediction model can be pre-trained using a variety of pre-training tasks for eventual downstream task training and utilization (e.g., interface prediction, interface generation, etc.).