公开(公告)号：US20210365281A1

公开(公告)日：2021-11-25

申请号：US17393276

申请日：2021-08-03

Applicant: Google LLC

Inventor： Prithvi Balaram ,  Nikhil Rao ,  Ian Baker ,  Adam Coimbra

IPC: G06F9/451 ,  G06F3/16 ,  G06F16/245

Abstract: Interface and mode selection for digital action execution is provided. For example, a system loads a script library embedded in an electronic resource. The system determines a historic level of engagement between a client computing device and one or more digital assistants. The system selects, based on a first property of the client computing device and the historic level of engagement, a type of digital interface. The system generates, based on the type of digital interface, a digital interface with the call-to-action. The system determines, responsive to an instruction to execute the call-to-action, a mode of execution. The system selects a digital assistant and a second client device to execute the call-to-action. The system transmits the call-to-action to the second client device for execution.

公开(公告)号：US11086959B2

公开(公告)日：2021-08-10

申请号：US16899016

申请日：2020-06-11

Applicant: Google LLC

Inventor： Adam Coimbra ,  Nikhil Rao ,  Hao Wu ,  Ian Baker ,  Anshul Gupta ,  Jay Akkad

IPC: G06F9/44 ,  G06F16/957 ,  G06F16/954 ,  G06F3/16 ,  G06F9/451 ,  G06F16/958 ,  G06F9/445 ,  G10L15/30 ,  H04L29/06 ,  G06F9/46 ,  G06F9/54

Abstract: Secure digital assistant integration with web pages is provided. The system receives an intent manifest data structure that maps actions of a digital assistant with link templates of an electronic resource developed by a third-party developer device. The system validates the electronic resource based on the intent manifest data structure. The system receives, from a data exchange component of an iframe of the electronic resource loaded by a client computing device, an identifier of the client computing device. The system receives a foreground state of the electronic resource from an onsite state sharing API. The system selects a data value for a parameter based on the foreground state and the intent manifest data structure. The system provides the data value. An authorization component generates an authorization prompt, receives input, and transmits the data value to an onsite intent execution API of the electronic resource to execute an action.

公开(公告)号：US20210235262A1

公开(公告)日：2021-07-29

申请号：US16466203

申请日：2019-05-06

Applicant: Google LLC

Inventor： Jay Akkad ,  Nikhil Rao ,  Anshul Gupta ,  David Wang ,  Ian Baker

IPC: H04W12/03 ,  H04L29/06 ,  G06F21/64 ,  G06F9/54 ,  G06F21/62 ,  G06F16/22

Abstract: Secure communication in mobile digital pages is provided. The system receives an electronic document and validates the electronic document for storage in a cache server. The system receives a request for the electronic document and provides it to a viewer component on a client computing device. The viewer component loads the electronic document in an iframe. The viewer component executes a runtime component to receive, via a secure communication channel, a tag from the electronic document. The system receives the tag and selects a data value for transmission to the viewer component. The viewer components provides the data value to cause the runtime component to execute an action with the data value.

公开(公告)号：US12106084B2

公开(公告)日：2024-10-01

申请号：US17840328

申请日：2022-06-14

Applicant: Google LLC

Inventor： Yan Huang ,  Nikhil Rao ,  Owen Lin ,  Ilya Firman ,  Anil Iyer

IPC: G06F8/60 ,  G06F9/54 ,  G06F11/36 ,  G06F16/9035 ,  G06F16/906 ,  G06F16/907

CPC classification number: G06F8/60 ,  G06F9/546 ,  G06F11/3604 ,  G06F11/362 ,  G06F11/3664 ,  G06F16/9035 ,  G06F16/906 ,  G06F16/907

Abstract: Analyzing or debugging applications is provided. The system identifies an action for an application provided by a developer. The system determines a first classification score based on historical execution of the action. The system generates a machine generated action for the application based on metadata associated with the application. The system determines a second classification score based on a comparison of the action with the machine generated action. The system selects, via a matching program, a second application that matches the application. The system determines a third classification score based on a comparison of an action approved for the second application with the action provided by the application developer. The system updates a delivery control parameter based on the first classification score, the second classification score and the third classification score. The system controls delivery of the application based on the delivery control parameter.

公开(公告)号：US12099778B2

公开(公告)日：2024-09-24

申请号：US17498542

申请日：2021-10-11

Applicant: GOOGLE LLC

Inventor： Quazi Hussain ,  Nikhil Rao

IPC: G06F3/16 ,  G10L15/22 ,  H04L43/0817 ,  H04L43/10 ,  H04L43/55

CPC classification number: G06F3/167 ,  G10L15/22 ,  H04L43/0817 ,  H04L43/10 ,  H04L43/55 ,  G10L2015/223

Abstract: The present disclosure is generally related to a data processing system to determine operational statuses of agents interfacing with digital assistant applications. The data processing system can access a log database. The log database can maintain response metrics for an agent service. Each response metric can correspond to a response by the agent service to a request generated by a digital assistant application responsive to an input audio signal. The data processing system can transmit a ping request to the agent service and can wait for a second response from the agent service. The data processing system can determine an operational status of the agent service based on the response metrics and on a time elapsed since the transmission of the ping request. The data processing system can cause the digital assistant application to perform an action based on the operational status.

公开(公告)号：US20240221747A1

公开(公告)日：2024-07-04

申请号：US18608271

申请日：2024-03-18

Applicant: GOOGLE LLC

Inventor： Dharma Muppalla ,  Nikhil Rao

IPC: G10L15/22 ,  G06F3/16 ,  G06F9/451 ,  G06F11/07 ,  G06F11/36 ,  G06F16/242 ,  G06F16/2455

CPC classification number: G10L15/22 ,  G06F3/167 ,  G06F9/453 ,  G06F11/0769 ,  G06F11/366 ,  G06F11/3664 ,  G06F11/3692 ,  G06F16/243 ,  G06F16/24565 ,  G10L2015/223 ,  G10L2015/225

Abstract: Validating actions in a digital assistant-based application is provided. The system identifies an application with a conversational interface. The system selects an action from an action repository and generates, via a natural language processor, a trigger phrase for input into the application. The system executes the application to process the trigger phrase to identify an action of the application. The system identifies a parameter used by the application to execute the action, and generates, based on the parameter and via execution of the conversational interface of the application, a first query responsive to the trigger phrase.
The system generates a first response to the first query for input into the application. The system determines, based on execution of the application to process the first response, a state of the application. The system evaluates the state to determine an error code and provide a notification based on the error code.

公开(公告)号：US20240185843A1

公开(公告)日：2024-06-06

申请号：US18440034

申请日：2024-02-13

Applicant: GOOGLE LLC

Inventor： Yan Huang ,  Nikhil Rao

IPC: G10L15/18 ,  G06F21/55 ,  G06F21/60 ,  G10L15/22 ,  G10L25/54 ,  G10L25/78

CPC classification number: G10L15/18 ,  G06F21/60 ,  G10L15/22 ,  G10L25/54 ,  G10L25/78 ,  G06F21/554 ,  G06F21/556 ,  G10L2015/223

Abstract: The present disclosure is generally related to a data processing system to detect potential exfiltration of audio data by agent applications can include a data processing system. The data processing system can identify, from an I/O record, an input received from the digital assistant application via a microphone of a client device, an output received from the agent application after the input, and a microphone status for the microphone. The data processing system can determine that the output is terminal based on the input and the output. The data processing system can identify the microphone status as in the enabled state subsequent to the input. The data processing system can determine that the agent application is unauthorized to access audio data acquired via the microphone of the client device based on determining that the output is terminal and identifying the microphone status as enabled.

公开(公告)号：US20240152369A1

公开(公告)日：2024-05-09

申请号：US18417681

申请日：2024-01-19

Applicant: GOOGLE LLC

Inventor： Prithvi Balaram ,  Nikhil Rao ,  Adam Coimbra ,  Ian Baker

IPC: G06F9/451 ,  G06F3/16 ,  G06F11/34 ,  G06F16/245

CPC classification number: G06F9/453 ,  G06F3/167 ,  G06F11/3438 ,  G06F16/245

Abstract: Interface and mode selection for digital action execution is provided. For example, a system loads a script library embedded in an electronic resource. The system determines a historic level of engagement between a client computing device and one or more digital assistants. The system selects, based on a first property of the client computing device and the historic level of engagement, a type of digital interface. The system generates, based on the type of digital interface, a digital interface with the call-to-action. The system determines, responsive to an instruction to execute the call-to-action, a mode of execution. The system selects a digital assistant and a second client device to execute the call-to-action. The system transmits the call-to-action to the second client device for execution.

公开(公告)号：US11922193B2

公开(公告)日：2024-03-05

申请号：US16643648

申请日：2020-02-28

Applicant: GOOGLE LLC

Inventor： Prithvi Balaram ,  Nikhil Rao ,  Ian Baker ,  Adam Coimbra

IPC: G06F9/451 ,  G06F3/16 ,  G06F11/34 ,  G06F16/245

CPC classification number: G06F9/453 ,  G06F3/167 ,  G06F11/3438 ,  G06F16/245

Abstract: Interface and mode selection for digital action execution is provided. For example, a system loads a script library embedded in an electronic resource. The system determines a historic level of engagement between a client computing device and one or more digital assistants. The system selects, based on a first property of the client computing device and the historic level of engagement, a type of digital interface. The system generates, based on the type of digital interface, a digital interface with the call-to-action. The system determines, responsive to an instruction to execute the call-to-action, a mode of execution. The system selects a digital assistant and a second client device to execute the call-to-action. The system transmits the call-to-action to the second client device for execution.

公开(公告)号：US11461221B2

公开(公告)日：2022-10-04

申请号：US16482187

申请日：2019-06-05

Applicant: Google LLC

Inventor： Dharma Muppalla ,  Nikhil Rao

IPC: G06F9/44 ,  G06F9/455 ,  G06F9/445 ,  G06F11/36 ,  G06F16/242 ,  G06F16/2455 ,  G06F3/16

Abstract: Validating actions in a digital assistant-based application is provided. The system identifies an application with a conversational interface. The system selects an action from an action repository and generates, via a natural language processor, a trigger phrase for input into the application. The system executes the application to process the trigger phrase to identify an action of the application. The system identifies a parameter used by the application to execute the action, and generates, based on the parameter and via execution of the conversational interface of the application, a first query responsive to the trigger phrase. The system generates a first response to the first query for input into the application. The system determines, based on execution of the application to process the first response, a state of the application. The system evaluates the state to determine an error code and provide a notification based on the error code.