公开(公告)号：US20210117150A1

公开(公告)日：2021-04-22

申请号：US16611461

申请日：2018-05-07

Applicant: Google LLC

Inventor： Quazi Hussain ,  Nikhil Rao

IPC: G06F3/16 ,  G10L15/22 ,  H04L12/26 ,  H04L12/24

Abstract: The present disclosure is generally related to a data processing system to determine operational statuses of agents interfacing with digital assistant applications. The data processing system can access a log database. The log database can maintain response metrics for an agent service. Each response metric can correspond to a response by the agent service to a request generated by a digital assistant application responsive to an input audio signal. The data processing system can transmit a ping request to the agent service and can wait for a second response from the agent service. The data processing system can determine an operational status of the agent service based on the response metrics and on a time elapsed since the transmission of the ping request. The data processing system can cause the digital assistant application to perform an action based on the operational status.

公开(公告)号：US10909971B2

公开(公告)日：2021-02-02

申请号：US16892951

申请日：2020-06-04

Applicant: Google LLC

Inventor： Yan Huang ,  Nikhil Rao

IPC: G10L15/00 ,  G10L15/18 ,  G10L25/54 ,  G10L15/22 ,  G10L25/78 ,  G06F21/60 ,  G06F21/55

Abstract: The present disclosure is generally related to a data processing system to detect potential exfiltration of audio data by agent applications can include a data processing system. The data processing system can identify, from an I/O record, an input received from the digital assistant application via a microphone of a client device, an output received from the agent application after the input, and a microphone status for the microphone. The data processing system can determine that the output is terminal based on the input and the output. The data processing system can identify the microphone status as in the enabled state subsequent to the input. The data processing system can determine that the agent application is unauthorized to access audio data acquired via the microphone of the client device based on determining that the output is terminal and identifying the microphone status as enabled.

公开(公告)号：US20200175967A1

公开(公告)日：2020-06-04

申请号：US16780299

申请日：2020-02-03

Applicant: Google LLC

Inventor： Yan Huang ,  Nikhil Rao

IPC: G10L15/18 ,  G06F21/60 ,  G10L25/78 ,  G10L15/22 ,  G10L25/54

Abstract: The present disclosure is generally related to a data processing system to detect potential exfiltration of audio data by agent applications can include a data processing system. The data processing system can identify, from an I/O record, an input received from the digital assistant application via a microphone of a client device, an output received from the agent application after the input, and a microphone status for the microphone. The data processing system can determine that the output is terminal based on the input and the output. The data processing system can identify the microphone status as in the enabled state subsequent to the input. The data processing system can determine that the agent application is unauthorized to access audio data acquired via the microphone of the client device based on determining that the output is terminal and identifying the microphone status as enabled.

公开(公告)号：US20240427581A1

公开(公告)日：2024-12-26

申请号：US18827258

申请日：2024-09-06

Applicant: GOOGLE LLC

Inventor： Yan Huang ,  Nikhil Rao ,  Owen Lin ,  Ilya Firman ,  Anil Iyer

IPC: G06F8/60 ,  G06F9/54 ,  G06F11/36 ,  G06F16/9035 ,  G06F16/906 ,  G06F16/907

Abstract: Analyzing or debugging applications is provided. The system identifies an action for an application provided by a developer. The system determines a first classification score based on historical execution of the action. The system generates a machine generated action for the application based on metadata associated with the application. The system determines a second classification score based on a comparison of the action with the machine generated action. The system selects, via a matching program, a second application that matches the application. The system determines a third classification score based on a comparison of an action approved for the second application with the action provided by the application developer. The system updates a delivery control parameter based on the first classification score, the second classification score and the third classification score. The system controls delivery of the application based on the delivery control parameter.

公开(公告)号：US20240402988A1

公开(公告)日：2024-12-05

申请号：US18806064

申请日：2024-08-15

Applicant: GOOGLE LLC

Inventor： Quazi Hussain ,  Nikhil Rao

IPC: G06F3/16 ,  G10L15/22 ,  H04L43/0817 ,  H04L43/10 ,  H04L43/55

Abstract: The present disclosure is generally related to a data processing system to determine operational statuses of agents interfacing with digital assistant applications. The data processing system can access a log database. The log database can maintain response metrics for an agent service. Each response metric can correspond to a response by the agent service to a request generated by a digital assistant application responsive to an input audio signal. The data processing system can transmit a ping request to the agent service and can wait for a second response from the agent service. The data processing system can determine an operational status of the agent service based on the response metrics and on a time elapsed since the transmission of the ping request. The data processing system can cause the digital assistant application to perform an action based on the operational status.

公开(公告)号：US12131100B2

公开(公告)日：2024-10-29

申请号：US17498542

申请日：2021-10-11

Applicant: GOOGLE LLC

Inventor： Quazi Hussain ,  Nikhil Rao

IPC: G06F3/16 ,  G10L15/22 ,  H04L43/0817 ,  H04L43/10 ,  H04L43/55

CPC classification number: G06F3/167 ,  G10L15/22 ,  H04L43/0817 ,  H04L43/10 ,  H04L43/55 ,  G10L2015/223

Abstract: The present disclosure is generally related to a data processing system to determine operational statuses of agents interfacing with digital assistant applications. The data processing system can access a log database. The log database can maintain response metrics for an agent service. Each response metric can correspond to a response by the agent service to a request generated by a digital assistant application responsive to an input audio signal. The data processing system can transmit a ping request to the agent service and can wait for a second response from the agent service. The data processing system can determine an operational status of the agent service based on the response metrics and on a time elapsed since the transmission of the ping request. The data processing system can cause the digital assistant application to perform an action based on the operational status.

公开(公告)号：US20230036983A1

公开(公告)日：2023-02-02

申请号：US17962991

申请日：2022-10-10

Applicant: GOOGLE LLC

Inventor： Jay Akkad ,  Nikhil Rao ,  Anshul Gupta ,  David Wang ,  Ian Baker ,  Neil Dhillon

IPC: H04W12/30 ,  H04W12/08 ,  G06F16/958 ,  G06F16/957 ,  G06F16/18 ,  G06F40/226 ,  G06F9/54 ,  G06F21/62 ,  H04L67/02 ,  G06F16/22 ,  H04W12/03 ,  G06F21/64 ,  H04L9/40 ,  H04L67/01

Abstract: Secure communication in mobile digital pages is provided. The system receives an electronic document and validates the electronic document for storage in a cache server. The system receives a request for the electronic document and provides it to a viewer component on a client computing device. The viewer component loads the electronic document in an iframe. The viewer component executes a runtime component to receive, via a secure communication channel, a tag from the electronic document. The system receives the tag and selects a data value for transmission to the viewer component. The viewer components provides the data value to cause the runtime component to execute an action with the data value.

公开(公告)号：US20230016967A1

公开(公告)日：2023-01-19

申请号：US17956480

申请日：2022-09-29

Applicant: GOOGLE LLC

Inventor： Dharma Muppalla ,  Nikhil Rao

IPC: G06F11/36 ,  G06F16/242 ,  G06F16/2455 ,  G06F3/16

Abstract: Validating actions in a digital assistant-based application is provided. The system identifies an application with a conversational interface. The system selects an action from an action repository and generates, via a natural language processor, a trigger phrase for input into the application. The system executes the application to process the trigger phrase to identify an action of the application. The system identifies a parameter used by the application to execute the action, and generates, based on the parameter and via execution of the conversational interface of the application, a first query responsive to the trigger phrase. The system generates a first response to the first query for input into the application. The system determines, based on execution of the application to process the first response, a state of the application. The system evaluates the state to determine an error code and provide a notification based on the error code.

公开(公告)号：US20220300600A1

公开(公告)日：2022-09-22

申请号：US17837914

申请日：2022-06-10

Applicant: Google LLC

Inventor： Adam Coimbra ,  Nikhil Rao ,  Hao Wu ,  Ian Baker ,  Anshul Gupta ,  Jay Akkad

IPC: G06F21/44 ,  G06F16/903

Abstract: Secure digital assistant integration with web pages is provided. The system receives an intent manifest data structure that maps actions of a digital assistant with link templates of an electronic resource developed by a third-party developer device. The system validates the electronic resource based on the intent manifest data structure. The system receives, from a data exchange component of an iframe of the electronic resource loaded by a client computing device, an identifier of the client computing device. The system receives a foreground state of the electronic resource from an onsite state sharing API. The system selects a data value for a parameter based on the foreground state and the intent manifest data structure. The system provides the data value. An authorization component generates an authorization prompt, receives input, and transmits the data value to an onsite intent execution API of the electronic resource to execute an action.

公开(公告)号：US11416229B2

公开(公告)日：2022-08-16

申请号：US16894101

申请日：2020-06-05

Applicant: GOOGLE LLC

Inventor： Yan Huang ,  Nikhil Rao ,  Owen Lin ,  Ilya Firman ,  Anil Iyer

IPC: G06F11/36 ,  G06F8/60 ,  G06F16/906 ,  G06F9/54 ,  G06F16/907 ,  G06F16/9035

Abstract: Analyzing or debugging applications is provided. The system identifies an action for an application provided by a developer. The system determines a first classification score based on historical execution of the action. The system generates a machine generated action for the application based on metadata associated with the application. The system determines a second classification score based on a comparison of the action with the machine generated action. The system selects, via a matching program, a second application that matches the application. The system determines a third classification score based on a comparison of an action approved for the second application with the action provided by the application developer. The system updates a delivery control parameter based on the first classification score, the second classification score and the third classification score. The system controls delivery of the application based on the delivery control parameter.