公开(公告)号：US11361066B2

公开(公告)日：2022-06-14

申请号：US16466213

申请日：2019-05-06

Applicant: Google LLC

Inventor： Adam Coimbra ,  Nikhil Rao ,  Hao Wu ,  Ian Baker ,  Anshul Gupta ,  Jay Akkad

IPC: G06F21/00 ,  G06F21/44 ,  G06F16/903

Abstract: Secure digital assistant integration with web pages is provided. The system receives an intent manifest data structure that maps actions of a digital assistant with link templates of an electronic resource developed by a third-party developer device. The system validates the electronic resource based on the intent manifest data structure. The system receives, from a data exchange component of an iframe of the electronic resource loaded by a client computing device, an identifier of the client computing device. The system receives a foreground state of the electronic resource from an onsite state sharing API. The system selects a data value for a parameter based on the foreground state and the intent manifest data structure. The system provides the data value. An authorization component generates an authorization prompt, receives input, and transmits the data value to an onsite intent execution API of the electronic resource to execute an action.

公开(公告)号：US11166155B2

公开(公告)日：2021-11-02

申请号：US16466203

申请日：2019-05-06

Applicant: Google LLC

Inventor： Jay Akkad ,  Nikhil Rao ,  Anshul Gupta ,  David Wang ,  Ian Baker

IPC: H04L29/06 ,  H04W12/03 ,  G06F16/22 ,  G06F9/54 ,  G06F21/62 ,  G06F21/64

Abstract: Secure communication in mobile digital pages is provided. The system receives an electronic document and validates the electronic document for storage in a cache server. The system receives a request for the electronic document and provides it to a viewer component on a client computing device. The viewer component loads the electronic document in an iframe. The viewer component executes a runtime component to receive, via a secure communication channel, a tag from the electronic document. The system receives the tag and selects a data value for transmission to the viewer component. The viewer components provides the data value to cause the runtime component to execute an action with the data value.

公开(公告)号：US20210294724A1

公开(公告)日：2021-09-23

申请号：US16482187

申请日：2019-06-05

Applicant: Google LLC

Inventor： Dharma Muppalla ,  Nikhil Rao

IPC: G06F11/36 ,  G06F16/2455 ,  G06F16/242 ,  G06F3/16

Abstract: Validating actions in a digital assistant-based application is provided. The system identifies an application with a conversational interface. The system selects an action from an action repository and generates, via a natural language processor, a trigger phrase for input into the application. The system executes the application to process the trigger phrase to identify an action of the application. The system identifies a parameter used by the application to execute the action, and generates, based on the parameter and via execution of the conversational interface of the application, a first query responsive to the trigger phrase. The system generates a first response to the first query for input into the application. The system determines, based on execution of the application to process the first response, a state of the application. The system evaluates the state to determine an error code and provide a notification based on the error code.

公开(公告)号：US20210272556A1

公开(公告)日：2021-09-02

申请号：US17321038

申请日：2021-05-14

Applicant: GOOGLE LLC

Inventor： Yan Huang ,  Nikhil Rao

IPC: G10L15/18 ,  G10L25/54 ,  G10L15/22 ,  G10L25/78 ,  G06F21/60

Abstract: The present disclosure is generally related to a data processing system to detect potential exfiltration of audio data by agent applications can include a data processing system. The data processing system can identify, from an I/O record, an input received from the digital assistant application via a microphone of a client device, an output received from the agent application after the input, and a microphone status for the microphone. The data processing system can determine that the output is terminal based on the input and the output. The data processing system can identify the microphone status as in the enabled state subsequent to the input. The data processing system can determine that the agent application is unauthorized to access audio data acquired via the microphone of the client device based on determining that the output is terminal and identifying the microphone status as enabled.

公开(公告)号：US11086644B1

公开(公告)日：2021-08-10

申请号：US16816794

申请日：2020-03-12

Applicant: GOOGLE LLC

Inventor： Prithvi Balaram ,  Nikhil Rao ,  Ian Baker ,  Adam Coimbra

IPC: G06F3/16 ,  G06F9/451 ,  G06F16/245

Abstract: Interface and mode selection for digital action execution is provided. For example, a system loads a script library embedded in an electronic resource. The system determines a historic level of engagement between a client computing device and one or more digital assistants. The system selects, based on a first property of the client computing device and the historic level of engagement, a type of digital interface. The system generates, based on the type of digital interface, a digital interface with the call-to-action. The system determines, responsive to an instruction to execute the call-to-action, a mode of execution. The system selects a digital assistant and a second client device to execute the call-to-action. The system transmits the call-to-action to the second client device for execution.

公开(公告)号：US10565985B1

公开(公告)日：2020-02-18

申请号：US16131439

申请日：2018-09-14

Applicant: Google LLC

Inventor： Yan Huang ,  Nikhil Rao

IPC: G10L15/00 ,  G10L15/18 ,  G10L25/54 ,  G10L15/22 ,  G10L25/78 ,  G06F21/60 ,  G06F21/55

Abstract: The present disclosure is generally related to a data processing system to detect potential exfiltration of audio data by agent applications can include a data processing system. The data processing system can identify, from an I/O record, an input received from the digital assistant application via a microphone of a client device, an output received from the agent application after the input, and a microphone status for the microphone. The data processing system can determine that the output is terminal based on the input and the output. The data processing system can identify the microphone status as in the enabled state subsequent to the input. The data processing system can determine that the agent application is unauthorized to access audio data acquired via the microphone of the client device based on determining that the output is terminal and identifying the microphone status as enabled.