公开(公告)号：US11967321B2

公开(公告)日：2024-04-23

申请号：US17538641

申请日：2021-11-30

Applicant: GOOGLE LLC

Inventor： Joseph Lange ,  Abhanshu Sharma ,  Adam Coimbra ,  Gökhan Bakir ,  Gabriel Taubman ,  Ilya Firman ,  Jindong Chen ,  James Stout ,  Marcin Nowak-Przygodzki ,  Reed Enger ,  Thomas Weedon Hume ,  Vishwath Mohan ,  Jacek Szmigiel ,  Yunfan Jin ,  Kyle Pedersen ,  Gilles Baechler

IPC: G10L15/22 ,  G06F3/16 ,  G06F40/247 ,  G06F40/30 ,  G10L15/18

CPC classification number: G10L15/22 ,  G06F3/167 ,  G06F40/247 ,  G06F40/30 ,  G10L15/1815 ,  G10L15/1822 ,  G10L2015/223 ,  G10L2015/228

Abstract: Implementations set forth herein relate to an automated assistant that can interact with applications that may not have been pre-configured for interfacing with the automated assistant. The automated assistant can identify content of an application interface of the application to determine synonymous terms that a user may speak when commanding the automated assistant to perform certain tasks. Speech processing operations employed by the automated assistant can be biased towards these synonymous terms when the user is accessing an application interface of the application. In some implementations, the synonymous terms can be identified in a responsive language of the automated assistant when the content of the application interface is being rendered in a different language. This can allow the automated assistant to operate as an interface between the user and certain applications that may not be rendering content in a native language of the user.

公开(公告)号：US20240281205A1

公开(公告)日：2024-08-22

申请号：US18650938

申请日：2024-04-30

Applicant: GOOGLE LLC

Inventor： Jacek Szmigiel ,  Joseph Lange

IPC: G06F3/16 ,  G06F3/0482 ,  G06F3/04847 ,  G06F3/04883 ,  G10L15/22

CPC classification number: G06F3/167 ,  G06F3/0482 ,  G06F3/04847 ,  G06F3/04883 ,  G10L15/22 ,  G10L2015/223

Abstract: Implementations set forth herein relate to an automated assistant that can control graphical user interface (GUI) elements via voice input using natural language understanding of GUI content in order to resolve ambiguity and allow for condensed GUI voice input requests. When a user is accessing an application that is rendering various GUI elements at a display interface, the automated assistant can operate to process actionable data corresponding to the GUI elements. The actionable data can be processed in order to determine a correspondence between GUI voice input requests to the automated assistant and at least one of the GUI elements rendered at the display interface. When a particular spoken utterance from the user is determined to correspond to multiple GUI elements, an indication of ambiguity can be rendered at the display interface in order to encourage the user to provide a more specific spoken utterance.

公开(公告)号：US11995379B2

公开(公告)日：2024-05-28

申请号：US17947359

申请日：2022-09-19

Applicant: GOOGLE LLC

Inventor： Jacek Szmigiel ,  Joseph Lange

IPC: G06F3/16 ,  G06F3/0482 ,  G06F3/04847 ,  G06F3/04883 ,  G10L15/22

CPC classification number: G06F3/167 ,  G06F3/0482 ,  G06F3/04847 ,  G06F3/04883 ,  G10L15/22 ,  G10L2015/223

Abstract: Implementations set forth herein relate to an automated assistant that can control graphical user interface (GUI) elements via voice input using natural language understanding of GUI content in order to resolve ambiguity and allow for condensed GUI voice input requests. When a user is accessing an application that is rendering various GUI elements at a display interface, the automated assistant can operate to process actionable data corresponding to the GUI elements. The actionable data can be processed in order to determine a correspondence between GUI voice input requests to the automated assistant and at least one of the GUI elements rendered at the display interface. When a particular spoken utterance from the user is determined to correspond to multiple GUI elements, an indication of ambiguity can be rendered at the display interface in order to encourage the user to provide a more specific spoken utterance.

公开(公告)号：US20230012852A1

公开(公告)日：2023-01-19

申请号：US17947359

申请日：2022-09-19

Applicant: GOOGLE LLC

Inventor： Jacek Szmigiel ,  Joseph Lange

IPC: G06F3/16 ,  G06F3/0482 ,  G06F3/04847 ,  G06F3/04883 ,  G10L15/22

Abstract: Implementations set forth herein relate to an automated assistant that can control graphical user interface (GUI) elements via voice input using natural language understanding of GUI content in order to resolve ambiguity and allow for condensed GUI voice input requests. When a user is accessing an application that is rendering various GUI elements at a display interface, the automated assistant can operate to process actionable data corresponding to the GUI elements. The actionable data can be processed in order to determine a correspondence between GUI voice input requests to the automated assistant and at least one of the GUI elements rendered at the display interface. When a particular spoken utterance from the user is determined to correspond to multiple GUI elements, an indication of ambiguity can be rendered at the display interface in order to encourage the user to provide a more specific spoken utterance.

公开(公告)号：US11449308B2

公开(公告)日：2022-09-20

申请号：US16972987

申请日：2019-08-12

Applicant: Google LLC

Inventor： Jacek Szmigiel ,  Joseph Lange

IPC: G06F3/16 ,  G06F3/0482 ,  G06F3/04847 ,  G06F3/04883 ,  G10L15/22

Abstract: Implementations set forth herein relate to an automated assistant that can control graphical user interface (GUI) elements via voice input using natural language understanding of GUI content in order to resolve ambiguity and allow for condensed GUI voice input requests. When a user is accessing an application that is rendering various GUI elements at a display interface, the automated assistant can operate to process actionable data corresponding to the GUI elements. The actionable data can be processed in order to determine a correspondence between GUI voice input requests to the automated assistant and at least one of the GUI elements rendered at the display interface. When a particular spoken utterance from the user is determined to correspond to multiple GUI elements, an indication of ambiguity can be rendered at the display interface in order to encourage the user to provide a more specific spoken utterance.

公开(公告)号：US20240274132A1

公开(公告)日：2024-08-15

申请号：US18642010

申请日：2024-04-22

Applicant: GOOGLE LLC

Inventor： Joseph Lange ,  Abhanshu Sharma ,  Adam Coimbra ,  Gökhan Bakir ,  Gabriel Taubman ,  IIya Firman ,  Jindong Chen ,  James Stout ,  Marcin Nowak-Przygodzki ,  Reed Enger ,  Thomas Weedon Hume ,  Vishwath Mohan ,  Jacek Szmigiel ,  Yunfan Jin ,  Kyle Pedersen ,  Gilles Baechler

IPC: G10L15/22 ,  G06F3/16 ,  G06F40/247 ,  G06F40/30 ,  G10L15/18

CPC classification number: G10L15/22 ,  G06F3/167 ,  G06F40/247 ,  G06F40/30 ,  G10L15/1815 ,  G10L15/1822 ,  G10L2015/223 ,  G10L2015/228

Abstract: Implementations set forth herein relate to an automated assistant that can interact with applications that may not have been pre-configured for interfacing with the automated assistant. The automated assistant can identify content of an application interface of the application to determine synonymous terms that a user may speak when commanding the automated assistant to perform certain tasks. Speech processing operations employed by the automated assistant can be biased towards these synonymous terms when the user is accessing an application interface of the application. In some implementations, the synonymous terms can be identified in a responsive language of the automated assistant when the content of the application interface is being rendered in a different language. This can allow the automated assistant to operate as an interface between the user and certain applications that may not be rendering content in a native language of the user.

公开(公告)号：US20210182018A1

公开(公告)日：2021-06-17

申请号：US16972987

申请日：2019-08-12

Applicant: Google LLC

Inventor： Jacek Szmigiel ,  Joseph Lange

IPC: G06F3/16 ,  G06F3/0484 ,  G06F3/0488 ,  G06F3/0482 ,  G10L15/22

Abstract: Implementations set forth herein relate to an automated assistant that can control graphical user interface (GUI) elements via voice input using natural language understanding of GUI content in order to resolve ambiguity and allow for condensed GUI voice input requests. When a user is accessing an application that is rendering various GUI elements at a display interface, the automated assistant can operate to process actionable data corresponding to the GUI elements. The actionable data can be processed in order to determine a correspondence between GUI voice input requests to the automated assistant and at least one of the GUI elements rendered at the display interface. When a particular spoken utterance from the user is determined to correspond to multiple GUI elements, an indication of ambiguity can be rendered at the display interface in order to encourage the user to provide a more specific spoken utterance.