Adapting automated assistant based on detected mouth movement and/or gaze

    公开(公告)号:US11614794B2

    公开(公告)日:2023-03-28

    申请号:US16606030

    申请日:2018-05-04

    Applicant: Google LLC

    Abstract: Adapting an automated assistant based on detecting: movement of a mouth of a user; and/or that a gaze of the user is directed at an assistant device that provides an automated assistant interface (graphical and/or audible) of the automated assistant. The detecting of the mouth movement and/or the directed gaze can be based on processing of vision data from one or more vision components associated with the assistant device, such as a camera incorporated in the assistant device. The mouth movement that is detected can be movement that is indicative of a user (to whom the mouth belongs) speaking.

    INVOKING AUTOMATED ASSISTANT FUNCTION(S) BASED ON DETECTED GESTURE AND GAZE

    公开(公告)号:US20230053873A1

    公开(公告)日:2023-02-23

    申请号:US17981181

    申请日:2022-11-04

    Applicant: GOOGLE LLC

    Abstract: Invoking one or more previously dormant functions of an automated assistant in response to detecting, based on processing of vision data from one or more vision components: (1) a particular gesture (e.g., of one or more “invocation gestures”) of a user; and/or (2) detecting that a gaze of the user is directed at an assistant device that provides an automated assistant interface (graphical and/or audible) of the automated assistant. For example, the previously dormant function(s) can be invoked in response to detecting the particular gesture, detecting that the gaze of the user is directed at an assistant device for at least a threshold amount of time, and optionally that the particular gesture and the directed gaze of the user co-occur or occur within a threshold temporal proximity of one another.

    METHODS AND SYSTEMS FOR ATTENDING TO A PRESENTING USER

    公开(公告)号:US20210334070A1

    公开(公告)日:2021-10-28

    申请号:US17370656

    申请日:2021-07-08

    Applicant: GOOGLE LLC

    Abstract: The various implementations described herein include methods, devices, and systems for attending to a presenting user. In one aspect, a method is performed at an electronic device that includes an image sensor, microphones, a display, processor(s), and memory. The device (1) obtains audio signals by concurrently receiving audio data at each microphone; (2) determines based on the obtained audio signals that a person is speaking in a vicinity of the device; (3) obtains video data from the image sensor; (4) determines via the video data that the person is not within a field of view of the image sensor; (5) reorients the electronic device based on differences in the received audio data; (6) after reorienting the electronic device, obtains second video data from the image sensor and determines that the person is within the field of view; and (7) attends to the person by directing the display toward the person.

    SELECTIVE DETECTION OF VISUAL CUES FOR AUTOMATED ASSISTANTS

    公开(公告)号:US20210232231A1

    公开(公告)日:2021-07-29

    申请号:US17229285

    申请日:2021-04-13

    Applicant: Google LLC

    Abstract: Techniques are described herein for reducing false positives in vision sensor-equipped assistant devices. In various implementations, initial image frame(s) may be obtained from vision sensor(s) of an assistant device and analyzed to classify a particular region of the initial image frames as being likely to contain visual noise. Subsequent image frame(s) obtained from the vision sensor(s) may then be analyzed to detect actionable user-provided visual cue(s), in a manner that reduces or eliminates false positives. In some implementations, no analysis may be performed on the particular region of the subsequent image frame(s). Additionally or alternatively, in some implementations, a first candidate visual cue detected within the particular region may be weighted less heavily than a second candidate visual cue detected elsewhere in the one or more subsequent image frames. An automated assistant may then take responsive action based on the detected actionable visual cue(s).

    Noise Mitigation for a Voice Interface Device

    公开(公告)号:US20210225387A1

    公开(公告)日:2021-07-22

    申请号:US17225693

    申请日:2021-04-08

    Applicant: GOOGLE LLC

    Inventor: Kenneth Mixter

    Abstract: A method at an electronic device with one or more microphones and a speaker, the electronic device configured to be responsive to any of a plurality of affordances including a voice-based affordance, includes determining background noise of an environment associated with the electronic device, and before detecting the voice-based affordance: determining whether the background noise would interfere with recognition of the hotword in voice inputs detected by the electronic device, and if so, indicating to a user to use an affordance other than the voice-based affordance.

    Noise Mitigation for a Voice Interface Device

    公开(公告)号:US20210201927A1

    公开(公告)日:2021-07-01

    申请号:US16995654

    申请日:2020-08-17

    Applicant: GOOGLE LLC

    Inventor: Kenneth Mixter

    Abstract: A method at an electronic device with one or more microphones and a speaker, the electronic device configured to be responsive to any of a plurality of affordances including a voice-based affordance, includes determining background noise of an environment associated with the electronic device, and before detecting the voice-based affordance: determining whether the background noise would interfere with recognition of the hotword in voice inputs detected by the electronic device, and if so, indicating to a user to use an affordance other than the voice-based affordance.

    Device leadership negotiation among voice interface devices

    公开(公告)号:US11024311B2

    公开(公告)日:2021-06-01

    申请号:US16786943

    申请日:2020-02-10

    Applicant: GOOGLE LLC

    Abstract: The various implementations described herein include methods and systems for determining device leadership among voice interface devices. In one aspect, a method is performed at a first electronic device of a plurality of electronic devices, each having microphones, a speaker, processors, and memory storing programs for execution by the processors. The first device detects a voice input. It determines a device state and a relevance of the voice input. It identifies a subset of electronic devices from the plurality to which the voice input is relevant. In accordance with a determination that the subset includes the first device, the first device determines a first score of a criterion associated with the voice input and receives second scores of the criterion from other devices in the subset. In accordance with a determination that the first score is higher than the second scores, the first device responds to the detected input.

    Conversation-Aware Proactive Notifications for a Voice Interface Device

    公开(公告)号:US20200302912A1

    公开(公告)日:2020-09-24

    申请号:US16894604

    申请日:2020-06-05

    Applicant: Google LLC

    Abstract: A method for proactive notifications in a voice interface device includes: receiving a first user voice request for an action with an future performance time; assigning the first user voice request to a voice assistant service for performance; subsequent to the receiving, receiving a second user voice request and in response to the second user voice request initiating a conversation with the user; and during the conversation: receiving a notification from the voice assistant service of performance of the action; triggering a first audible announcement to the user to indicate a transition from the conversation and interrupting the conversation; triggering a second audible announcement to the user to indicate performance of the action; and triggering a third audible announcement to the user to indicate a transition back to the conversation and rejoining the conversation.

    Device leadership negotiation among voice interface devices

    公开(公告)号:US10559306B2

    公开(公告)日:2020-02-11

    申请号:US16159339

    申请日:2018-10-12

    Applicant: GOOGLE LLC

    Abstract: The various implementations described herein include methods and systems for determining device leadership among voice interface devices. In one aspect, a method is performed at a first electronic device of a plurality of electronic devices, each having microphones, a speaker, processors, and memory storing programs for execution by the processors. The first device detects a voice input. It determines a device state and a relevance of the voice input. It identifies a subset of electronic devices from the plurality to which the voice input is relevant. In accordance with a determination that the subset includes the first device, the first device determines a first score of a criterion associated with the voice input and receives second scores of the criterion from other devices in the subset. In accordance with a determination that the first score is higher than the second scores, the first device responds to the detected input.

Patent Agency Ranking