公开(公告)号：US20240330362A1

公开(公告)日：2024-10-03

申请号：US18555814

申请日：2022-10-25

申请人： GOOGLE LLC

发明人： Ruofei Du ,  Alex Olwal ,  Xingyu Liu

IPC分类号： G06F16/58 ,  G02B27/01 ,  G10L15/26

CPC分类号： G06F16/58 ,  G02B27/017 ,  G10L15/26 ,  G02B2027/0178

摘要： Methods and devices are provided where a device may receive audio data via a sensor of a computing device. The device may convert the audio data to text and extract a portion of the text. The device may input the portion of the text to a neural network-based language model to obtain at least one of a type of visual images, a source of the visual images, a content of the visual images, or a confidence score for the visual images. The device may determine at least one visual image based on at least one of the type of the visual images, the source of the visual images, the content of the visual images, or the confidence score for each of the visual images. The at least one visual image may be output on a display of the computing device to supplement the audio data and facilitate a communication.

公开(公告)号：US20230367116A1

公开(公告)日：2023-11-16

申请号：US17663444

申请日：2022-05-16

申请人： GOOGLE LLC

发明人： Ruofei Du ,  Maksym Dzitsiuk

IPC分类号： G02B27/00 ,  G08B25/01

CPC分类号： G02B27/0093 ,  G08B25/016 ,  G02B2027/0178

摘要： A head-mounted device, such as augmented reality glasses, can be configured to track screen time. The head-mounted device can advantageously track screen time across a variety of devices whether or not they are configured to such measurements. Various modes for tracking the screen time may be used based on accuracy, power consumption, and privacy. The head-mounted device may track in a mode based on a battery level, a device setting or a user input. The head-mounted device may accumulate screen times in a database and in some implementations, can link screen times with devices in the database based on device information gathered by the head-mounted device during the tracking process. The head-mounted device may be configured to generate and display an alert based on one or more rules about screen time.

公开(公告)号：US20230206567A1

公开(公告)日：2023-06-29

申请号：US17998736

申请日：2020-05-29

申请人： GOOGLE LLC

发明人： David Kim ,  Ruofei Du

IPC分类号： G06T19/00 ,  G06T17/20 ,  G06T7/50 ,  G06T15/06 ,  G06T15/60

CPC分类号： G06T19/006 ,  G06T17/20 ,  G06T7/50 ,  G06T15/06 ,  G06T15/60 ,  G06T2207/10028 ,  G06T2200/28 ,  G06T2215/12 ,  G06T2215/16 ,  G06T2210/21

摘要： Techniques of introducing virtual objects into a physical environment of AR system include displacing vertices of a mesh representing the physical environment based on a live depth map. For example, an AR system generates a mesh template, i.e., an initial mesh with vertices that represents a physical environment and a depth map that indicates a geometry of real objects within the physical environment. The AR system is configured to represent the real objects in the physical environment by displacing the vertices of the mesh based on depth values of the depth map and parameter values of a pinhole camera model. The depth values may be taken from the perspective of an illumination source in the physical environment.

公开(公告)号：US12061738B2

公开(公告)日：2024-08-13

申请号：US18326854

申请日：2023-05-31

申请人： GOOGLE LLC

发明人： Ruofei Du ,  Maksym Dzitsiuk

IPC分类号： G06F3/01 ,  G06T7/70 ,  G06V10/77 ,  G06V20/50

CPC分类号： G06F3/013 ,  G06T7/70 ,  G06V10/7715 ,  G06V20/50

摘要： A method including determining a gaze direction of a user of a wearable device, capturing an image using a forward-looking camera of the wearable device, detecting a surroundings of the user based on the image, determining whether or not the user is distracted based on the gaze direction and the surroundings, and in response to determining the user is distracted, causing an operation to be performed on the wearable device, the operation configured to cause the user to change the user's attention.

公开(公告)号：US20240212184A1

公开(公告)日：2024-06-27

申请号：US18555059

申请日：2021-04-30

申请人： Google LLC

发明人： Ruofei Du ,  David Li ,  Danhang Tang ,  Yinda Zhang

IPC分类号： G06T7/55 ,  G06T5/77 ,  G06T7/181 ,  G06T15/00 ,  G06T17/20

CPC分类号： G06T7/55 ,  G06T5/77 ,  G06T7/181 ,  G06T15/00 ,  G06T17/20 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221

摘要： A method including predicting a stereo depth associated with a first panoramic image and a second panoramic image, the first panoramic image and the second panoramic image being captured with a time interlude between the capture of the first panoramic image and the second panoramic image, generating a first mesh representation based on the first panoramic image and a stereo depth corresponding to the first panoramic image, generating a second mesh representation based on the second panoramic image and a stereo depth corresponding to the second panoramic image, and synthesizing a third panoramic image based on fusing the first mesh representation with the second mesh representation.

公开(公告)号：US20240144523A1

公开(公告)日：2024-05-02

申请号：US18495483

申请日：2023-10-26

申请人： GOOGLE LLC

发明人： Ruofei Du ,  Ananta Narayanan Balaji ,  David Kim ,  Clayton Merrill Kimber

IPC分类号： G06T7/73 ,  G06T7/80

CPC分类号： G06T7/73 ,  G06T7/80 ,  G06T2207/10012 ,  G06T2207/10048 ,  G06T2207/20084 ,  G06T2207/30204

摘要： According to an aspect, a method may include receiving two-dimensional (2D) positions of at least one of a first reflective marker or a second reflective marker of a physical component, estimating a three-dimensional (3D) position of the first reflective marker and a 3D position of the second reflective marker based on the 2D positions, and computing an orientation of the physical component in 3D space based on the 3D position of the first reflective marker, the 3D position of the second reflective marker, and positioning information of the first and second reflective markers in the physical component.

公开(公告)号：US20230393657A1

公开(公告)日：2023-12-07

申请号：US18326854

申请日：2023-05-31

申请人： GOOGLE LLC

发明人： Ruofei Du ,  Maksym Dzitsiuk

IPC分类号： G06F3/01 ,  G06V20/50 ,  G06V10/77 ,  G06T7/70

CPC分类号： G06F3/013 ,  G06V20/50 ,  G06V10/7715 ,  G06T7/70

摘要： A method including determining a gaze direction of a user of a wearable device, capturing an image using a forward-looking camera of the wearable device, detecting a surroundings of the user based on the image, determining whether or not the user is distracted based on the gaze direction and the surroundings, and in response to determining the user is distracted, causing an operation to be performed on the wearable device, the operation configured to cause the user to change the user's attention.

公开(公告)号：US20230305672A1

公开(公告)日：2023-09-28

申请号：US17656818

申请日：2022-03-28

申请人： Google LLC

发明人： Ruofei Du ,  Alex Olwal ,  Mathieu Simon Le Goc ,  David Kim ,  Danhang Tang

IPC分类号： G06F3/04815 ,  G10L15/22 ,  G02B27/01

CPC分类号： G06F3/04815 ,  G10L15/22 ,  G02B27/0172 ,  G02B27/0176 ,  G02B2027/0178 ,  G10L2015/223

摘要： Systems and methods are provided in which physical objects in the ambient environment can function as user interface implements in an augmented reality environment. A physical object detected within a field of view of a camera of a computing device may be designated as a user interface implement in response to a user command. User interfaces may be attached to the designated physical object, to provide a tangible user interface implement for user interaction with the augmented reality environment.

公开(公告)号：US20220414834A1

公开(公告)日：2022-12-29

申请号：US17756700

申请日：2020-05-26

申请人： Google LLC

发明人： Ruofei Du ,  David Kim

IPC分类号： G06T5/00 ,  G06T7/50 ,  G06T7/13

摘要： A method including receiving an image as a portion of a real-world space, placing an anchor on the image, determining a position of the anchor, determine a depth associated with the position of the anchor, applying an image editing algorithm based on the depth to the captured image, and rendering the edited image.

公开(公告)号：US20240290025A1

公开(公告)日：2024-08-29

申请号：US18588948

申请日：2024-02-27

申请人： GOOGLE LLC

发明人： Yinda Zhang ,  Sean Ryan Francesco Fanello ,  Ziqian Bai ,  Feitong Tan ,  Zeng Huang ,  Kripasindhu Sarkar ,  Danhang Tang ,  Di Qiu ,  Abhimitra Meka ,  Ruofei Du ,  Mingsong Dou ,  Sergio Orts Escolano ,  Rohit Kumar Pandey ,  Thabo Beeler

IPC分类号： G06T13/40 ,  G06T7/90 ,  G06T17/20 ,  G06V10/44

CPC分类号： G06T13/40 ,  G06T7/90 ,  G06T17/20 ,  G06V10/44 ,  G06T2207/10024 ,  G06T2207/20084

摘要： A method comprises receiving a first sequence of images of a portion of a user, the first sequence of images being monocular images; generating an avatar based on the first sequence of images, the avatar being based on a model including a feature vector associated with a vertex; receiving a second sequence of images of the portion of the user; and based on the second sequence of images, modifying the avatar with a displacement of the vertex to represent a gesture of the avatar.