公开(公告)号：US20250085557A1

公开(公告)日：2025-03-13

申请号：US18464019

申请日：2023-09-08

Applicant: GOOGLE LLC

Inventor： Mahdi Tayarani Najaran ,  Ruofei Du

IPC: G02B27/01 ,  G02B27/00

Abstract: In described techniques, a depth of an object may be determined with respect to a head-mounted device (HMD). A gaze duration with respect to the object may be determined, relative to a gaze duration threshold. Then, based on the depth and on the gaze duration exceeding the gaze duration threshold, virtual content may be rendered at a virtual depth with respect to the HMD that is different than the depth.

公开(公告)号：US20240412495A1

公开(公告)日：2024-12-12

申请号：US18736910

申请日：2024-06-07

Applicant: Google LLC

Inventor： Ruofei Du ,  Xingyu Liu

IPC: G06V10/80 ,  G06V10/25 ,  G06V10/94 ,  G06V20/20 ,  G06V20/50

Abstract: A method comprises determining, by a head-mounted device, a limitation of a user based on a visual input received from a camera included in the head-mounted device; determining an output communication channel and an input communication channel based on the limitation of the user, sending a notification to the user via the output communication channel; and receiving an input from the user via the input communication channel.

公开(公告)号：US12167082B2

公开(公告)日：2024-12-10

申请号：US17933631

申请日：2022-09-20

Applicant: Google LLC

Inventor： Alex Olwal ,  Ruofei Du

IPC: G06F3/16 ,  G06F3/01 ,  G06T19/00 ,  H04N21/431 ,  H04N21/442

Abstract: Systems and methods are related to tracking an attention of a user with respect to content presented on a virtual screen, detecting a defocus event associated with a first region of the content, and determining a next focus event associated with a second region of the content. The determination can be based at least in part on the defocus event and on the tracked attention of the user. The systems and methods can include generating, based on the determined next focus event, a marker for differentiating the second region of the content from a remainder of the content, and in response to detecting a refocus event associated with the virtual screen, triggering execution of the marker associated with the second region of the content.

公开(公告)号：US20240330362A1

公开(公告)日：2024-10-03

申请号：US18555814

申请日：2022-10-25

Applicant: GOOGLE LLC

Inventor： Ruofei Du ,  Alex Olwal ,  Xingyu Liu

IPC: G06F16/58 ,  G02B27/01 ,  G10L15/26

CPC classification number: G06F16/58 ,  G02B27/017 ,  G10L15/26 ,  G02B2027/0178

Abstract: Methods and devices are provided where a device may receive audio data via a sensor of a computing device. The device may convert the audio data to text and extract a portion of the text. The device may input the portion of the text to a neural network-based language model to obtain at least one of a type of visual images, a source of the visual images, a content of the visual images, or a confidence score for the visual images. The device may determine at least one visual image based on at least one of the type of the visual images, the source of the visual images, the content of the visual images, or the confidence score for each of the visual images. The at least one visual image may be output on a display of the computing device to supplement the audio data and facilitate a communication.

公开(公告)号：US20230367116A1

公开(公告)日：2023-11-16

申请号：US17663444

申请日：2022-05-16

Applicant: GOOGLE LLC

Inventor： Ruofei Du ,  Maksym Dzitsiuk

IPC: G02B27/00 ,  G08B25/01

CPC classification number: G02B27/0093 ,  G08B25/016 ,  G02B2027/0178

Abstract: A head-mounted device, such as augmented reality glasses, can be configured to track screen time. The head-mounted device can advantageously track screen time across a variety of devices whether or not they are configured to such measurements. Various modes for tracking the screen time may be used based on accuracy, power consumption, and privacy. The head-mounted device may track in a mode based on a battery level, a device setting or a user input. The head-mounted device may accumulate screen times in a database and in some implementations, can link screen times with devices in the database based on device information gathered by the head-mounted device during the tracking process. The head-mounted device may be configured to generate and display an alert based on one or more rules about screen time.

公开(公告)号：US20230206567A1

公开(公告)日：2023-06-29

申请号：US17998736

申请日：2020-05-29

Applicant: GOOGLE LLC

Inventor： David Kim ,  Ruofei Du

IPC: G06T19/00 ,  G06T17/20 ,  G06T7/50 ,  G06T15/06 ,  G06T15/60

CPC classification number: G06T19/006 ,  G06T17/20 ,  G06T7/50 ,  G06T15/06 ,  G06T15/60 ,  G06T2207/10028 ,  G06T2200/28 ,  G06T2215/12 ,  G06T2215/16 ,  G06T2210/21

Abstract: Techniques of introducing virtual objects into a physical environment of AR system include displacing vertices of a mesh representing the physical environment based on a live depth map. For example, an AR system generates a mesh template, i.e., an initial mesh with vertices that represents a physical environment and a depth map that indicates a geometry of real objects within the physical environment. The AR system is configured to represent the real objects in the physical environment by displacing the vertices of the mesh based on depth values of the depth map and parameter values of a pinhole camera model. The depth values may be taken from the perspective of an illumination source in the physical environment.

公开(公告)号：US20240380981A1

公开(公告)日：2024-11-14

申请号：US18316710

申请日：2023-05-12

Applicant: GOOGLE LLC

Inventor： Ruofei Du ,  Alex Olwal

IPC: H04N23/73 ,  G02B27/01 ,  G06F3/01 ,  G06T1/20 ,  G06T5/20 ,  G06V10/25 ,  G06V10/60 ,  H04N23/698

Abstract: Systems and methods are disclosed that address the need for adaptive exposure within high dynamic range (HDR) images. Solutions can leverage recent advances in the use of virtual reality (VR) headsets and Augmented Reality (AR) displays equipped with infrared (IR) eye tracking devices. A gaze vector determined by the eye tracking device identifies one or more fixation points on the image that corresponds to an area where there exists a faulty exposure. The exposure around the fixation point can be adaptively corrected using image processing techniques. Using spatial adaptive exposure, the resulting image, a type of foveated image, can be rendered on a low dynamic range (LDR) display with sufficient detail.

公开(公告)号：US12061738B2

公开(公告)日：2024-08-13

申请号：US18326854

申请日：2023-05-31

Applicant: GOOGLE LLC

Inventor： Ruofei Du ,  Maksym Dzitsiuk

IPC: G06F3/01 ,  G06T7/70 ,  G06V10/77 ,  G06V20/50

CPC classification number: G06F3/013 ,  G06T7/70 ,  G06V10/7715 ,  G06V20/50

Abstract: A method including determining a gaze direction of a user of a wearable device, capturing an image using a forward-looking camera of the wearable device, detecting a surroundings of the user based on the image, determining whether or not the user is distracted based on the gaze direction and the surroundings, and in response to determining the user is distracted, causing an operation to be performed on the wearable device, the operation configured to cause the user to change the user's attention.

公开(公告)号：US20240212184A1

公开(公告)日：2024-06-27

申请号：US18555059

申请日：2021-04-30

Applicant: Google LLC

Inventor： Ruofei Du ,  David Li ,  Danhang Tang ,  Yinda Zhang

IPC: G06T7/55 ,  G06T5/77 ,  G06T7/181 ,  G06T15/00 ,  G06T17/20

CPC classification number: G06T7/55 ,  G06T5/77 ,  G06T7/181 ,  G06T15/00 ,  G06T17/20 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221

Abstract: A method including predicting a stereo depth associated with a first panoramic image and a second panoramic image, the first panoramic image and the second panoramic image being captured with a time interlude between the capture of the first panoramic image and the second panoramic image, generating a first mesh representation based on the first panoramic image and a stereo depth corresponding to the first panoramic image, generating a second mesh representation based on the second panoramic image and a stereo depth corresponding to the second panoramic image, and synthesizing a third panoramic image based on fusing the first mesh representation with the second mesh representation.

公开(公告)号：US20240144523A1

公开(公告)日：2024-05-02

申请号：US18495483

申请日：2023-10-26

Applicant: GOOGLE LLC

Inventor： Ruofei Du ,  Ananta Narayanan Balaji ,  David Kim ,  Clayton Merrill Kimber

IPC: G06T7/73 ,  G06T7/80

CPC classification number: G06T7/73 ,  G06T7/80 ,  G06T2207/10012 ,  G06T2207/10048 ,  G06T2207/20084 ,  G06T2207/30204

Abstract: According to an aspect, a method may include receiving two-dimensional (2D) positions of at least one of a first reflective marker or a second reflective marker of a physical component, estimating a three-dimensional (3D) position of the first reflective marker and a 3D position of the second reflective marker based on the 2D positions, and computing an orientation of the physical component in 3D space based on the 3D position of the first reflective marker, the 3D position of the second reflective marker, and positioning information of the first and second reflective markers in the physical component.