公开(公告)号：US20230186575A1

公开(公告)日：2023-06-15

申请号：US17998903

申请日：2020-05-22

Applicant: GOOGLE LLC

Inventor： Eric Turner ,  Keisuke Tateno ,  Konstantine Nicholas John Tsotsos ,  Adarsh Prakash Murthy Kowdle ,  Vaibhav Gupta ,  Ambrus Csaszar

IPC: G06T19/00 ,  G06T1/60

CPC classification number: G06T19/006 ,  G06T1/60

Abstract: A method including receiving a first depth image associated with a first frame at a first time of an augmented reality (AR) application, the first depth image representing at least a first portion of a real-world space storing the first depth image receiving a second depth image associated with a second frame at a second time, after the first time, of the AR application, the second depth image representing at least a second portion of the real-world space generating a real-world image by blending, at least, the stored first depth image with the second depth image receiving a rendered AR object combining the AR object in the real-world image and displaying the real-world image combined with the AR object.

公开(公告)号：US20230410344A1

公开(公告)日：2023-12-21

申请号：US17807041

申请日：2022-06-15

Applicant: GOOGLE LLC

Inventor： Eric Turner ,  Idris Syed Aleem ,  Sushant Umesh Kulkarni ,  Rees Anwyl Samuel Simmons ,  Ryan Christopher DuToit

IPC: G06T7/60 ,  G06T7/73

CPC classification number: G06T7/60 ,  G06T2207/30201 ,  G06T7/74

Abstract: A system and method of detecting scale from image data obtained by a computing device together with position and orientation of the computing device is provided. The scale may be used for the virtual selection and/or fitting of a wearable device such as a head mounted wearable device and other such wearable devices. The system and method may include capturing a series of frames of image data, and detecting one or more fixed features in the series of frames of image data. Position and orientation data associated with the capture of the image data is combined with the position data related to the one or more fixed features to set a scale that is applicable to the image data. Scale may be determined without the use of a reference object having known scale, specialized equipment, and the like.

公开(公告)号：US20230236219A1

公开(公告)日：2023-07-27

申请号：US17648572

申请日：2022-01-21

Applicant: GOOGLE LLC

Inventor： Yunwen Zhou ,  Ryan Christopher DuToit ,  Abhishek Kar ,  Konstantine Nicholas John Tsotsos ,  Eric Turner ,  Chao Guo

IPC: G01P21/00 ,  G06T7/50 ,  G06T7/80 ,  G06T7/33 ,  G06T7/20 ,  G01P13/00 ,  G01P15/08

CPC classification number: G01P21/00 ,  G06T7/50 ,  G06T7/80 ,  G06T7/33 ,  G06T7/20 ,  G01P13/00 ,  G01P15/08 ,  G06T2207/20084

Abstract: Disclosed is a method including receiving a depth map estimated using data based on image and data received from a movement sensor as input, generating an alignment parameter based on the depth map, adding the alignment parameter to a pre-calibration state to define a user operational calibration state, generating scale parameters and shift parameters based on features associated with the data received from the image and movement sensor, and calibrating the image and movement sensor based on the user operational calibration state, the scale parameters and the shift parameters.

公开(公告)号：US11145075B2

公开(公告)日：2021-10-12

申请号：US16767401

申请日：2019-10-04

Applicant: Google LLC

Inventor： Julien Valentin ,  Onur G. Guleryuz ,  Mira Leung ,  Maksym Dzitsiuk ,  Jose Pascoal ,  Mirko Schmidt ,  Christoph Rhemann ,  Neal Wadhwa ,  Eric Turner ,  Sameh Khamis ,  Adarsh Prakash Murthy Kowdle ,  Ambrus Csaszar ,  João Manuel Castro Afonso ,  Jonathan T. Barron ,  Michael Schoenberg ,  Ivan Dryanovski ,  Vivek Verma ,  Vladimir Tankovich ,  Shahram Izadi ,  Sean Ryan Francesco Fanello ,  Konstantine Nicholas John Tsotsos

IPC: G06T7/579 ,  G06T19/00 ,  G06T7/571

Abstract: A handheld user device includes a monocular camera to capture a feed of images of a local scene and a processor to select, from the feed, a keyframe and perform, for a first image from the feed, stereo matching using the first image, the keyframe, and a relative pose based on a pose associated with the first image and a pose associated with the keyframe to generate a sparse disparity map representing disparities between the first image and the keyframe. The processor further is to determine a dense depth map from the disparity map using a bilateral solver algorithm, and process a viewfinder image generated from a second image of the feed with occlusion rendering based on the depth map to incorporate one or more virtual objects into the viewfinder image to generate an AR viewfinder image. Further, the processor is to provide the AR viewfinder image for display.

公开(公告)号：US10553016B2

公开(公告)日：2020-02-04

申请号：US15813558

申请日：2017-11-15

Applicant: Google LLC

Inventor： Eric Turner ,  Damien Saint-Macary ,  Behnam Bastani ,  Haomiao Jiang

IPC: G06T15/00 ,  G06T15/20 ,  G02B27/01 ,  G06F3/01 ,  G06T19/00 ,  G06F3/0481

Abstract: A display device, such as a head mounted device (HMD), displays a virtual scene. The display device includes a motion tracker for detecting rotation and/or translation of the display device. The display device also includes a processor that is configured to determine, in response to the detected, an orientation of the display device relative to a plurality of world-aligned viewing frustums that are stationary relative to the virtual scene. The processor is also configured to identify a set of those world-aligned viewing frustums that overlap with an output field of view. The processor is further configured to render pixels of the set of those world-aligned viewing frustums that overlap with an output field of view and upsample the rendered pixels to generate values of display pixels for presentation by the display device.

公开(公告)号：US10564715B2

公开(公告)日：2020-02-18

申请号：US15351196

申请日：2016-11-14

Applicant: Google LLC

Inventor： Mark Spitzer ,  Carlin Vieri ,  Eric Turner ,  Behnam Bastani

IPC: G06F3/01 ,  G02B27/00 ,  G06F3/147 ,  G09G5/391 ,  G09G5/00 ,  G06T1/20 ,  G06T3/40

Abstract: A foveated display system includes a rendering device including at least one graphics processing unit (GPU) to render a foveal region and a peripheral region of a first image, wherein the foveal region has a higher resolution than the peripheral region. The system further includes a display device coupled to the rendering device via at least one physical layer. The display device includes a pixel array and a display controller coupled to the pixel array. The display controller includes a scaling component to upscale the first peripheral region to generate a scaled first peripheral region and a blending component to blend the foveal region with the scaled first peripheral region to generate a second image.

公开(公告)号：US20230274491A1

公开(公告)日：2023-08-31

申请号：US18001659

申请日：2021-09-01

Applicant: GOOGLE LLC

Inventor： Eric Turner ,  Adarsh Prakash Murthy Kowdle ,  Bicheng Luo ,  Juan David Hincapie Ramos

IPC: G06T15/04 ,  G06T7/50

CPC classification number: G06T15/04 ,  G06T7/50

Abstract: A method including receiving (S605) a request for a depth map, generating (S625) a hybrid depth map based on a device depth map (110) and downloaded depth information (105), and responding (S630) to the request for the depth map with the hybrid depth map (415). The device depth map (110) can be depth data captured on a user device (515) using sensors and/or software. The downloaded depth information (105) can be associated with depth data, map data, image data, and/or the like stored on a remote (to the user device) server (505).