公开(公告)号：US20240265605A1

公开(公告)日：2024-08-08

申请号：US18165779

申请日：2023-02-07

Applicant: GOOGLE LLC

Inventor： Ruofei Du ,  Xingyu Liu

IPC: G06T13/20 ,  G06T13/40 ,  G06T13/80 ,  G10L25/63

CPC classification number: G06T13/205 ,  G06T13/40 ,  G06T13/80 ,  G10L25/63

Abstract: A system and method may receive audio signal information associated with a user. An expression prediction may be determined by executing an expression determination model using the audio signal information as input. An avatar animation may be generated based on the expression prediction, where the avatar animation includes non-verbal expression representing the expression prediction.

公开(公告)号：US20240412495A1

公开(公告)日：2024-12-12

申请号：US18736910

申请日：2024-06-07

Applicant: Google LLC

Inventor： Ruofei Du ,  Xingyu Liu

IPC: G06V10/80 ,  G06V10/25 ,  G06V10/94 ,  G06V20/20 ,  G06V20/50

Abstract: A method comprises determining, by a head-mounted device, a limitation of a user based on a visual input received from a camera included in the head-mounted device; determining an output communication channel and an input communication channel based on the limitation of the user, sending a notification to the user via the output communication channel; and receiving an input from the user via the input communication channel.

公开(公告)号：US20240330362A1

公开(公告)日：2024-10-03

申请号：US18555814

申请日：2022-10-25

Applicant: GOOGLE LLC

Inventor： Ruofei Du ,  Alex Olwal ,  Xingyu Liu

IPC: G06F16/58 ,  G02B27/01 ,  G10L15/26

CPC classification number: G06F16/58 ,  G02B27/017 ,  G10L15/26 ,  G02B2027/0178

Abstract: Methods and devices are provided where a device may receive audio data via a sensor of a computing device. The device may convert the audio data to text and extract a portion of the text. The device may input the portion of the text to a neural network-based language model to obtain at least one of a type of visual images, a source of the visual images, a content of the visual images, or a confidence score for the visual images. The device may determine at least one visual image based on at least one of the type of the visual images, the source of the visual images, the content of the visual images, or the confidence score for each of the visual images. The at least one visual image may be output on a display of the computing device to supplement the audio data and facilitate a communication.