公开(公告)号：US20230110964A1

公开(公告)日：2023-04-13

申请号：US18065436

申请日：2022-12-13

Applicant: Google LLC

Inventor： Jason Todd Spencer ,  Seth Raphael ,  Sofien Bouaziz

IPC: G06F3/01 ,  G06V10/75

Abstract: A wearable computing device includes a frame, a camera mounted on the frame so as to capture images of an environment outside of the wearable computing device, a display device mounted on the frame so as to display the images captured by the camera, and at least one eye gaze tracking device mounted on the frame so as to track a gaze directed at the images displayed by the display device. In response to the detection of a fixation of the gaze on the display of images, the system may identify a pixel area corresponding to a fixation point of the fixation gaze on the display of images. The system may identify an object in the ambient environment corresponding to the identified pixel area, and set the identified object as a selected object for user interaction.

公开(公告)号：US20220300082A1

公开(公告)日：2022-09-22

申请号：US17249966

申请日：2021-03-19

Applicant: GOOGLE LLC

Inventor： Dongeek Shin ,  Shahram Izadi ,  David Kim ,  Sofien Bouaziz ,  Steven Benjamin Goldberg ,  Ivan Poupyrev ,  Shwetak N. Patel

IPC: G06F3/01 ,  G06K9/00 ,  G06F1/16

Abstract: Techniques of identifying gestures include detecting and classifying inner-wrist muscle motions at a user's wrist using micron-resolution radar sensors. For example, a user of an AR system may wear a band around their wrist. When the user makes a gesture to manipulate a virtual object in the AR system as seen in a head-mounted display (HMD), muscles and ligaments in the user's wrist make small movements on the order of 1-3 mm. The band contains a small radar device that has a transmitter and a number of receivers (e.g., three) of electromagnetic (EM) radiation on a chip (e.g., a Soli chip. This radiation reflects off the wrist muscles and ligaments and is received by the receivers on the chip in the band. The received reflected signal, or signal samples, are then sent to processing circuitry for classification to identify the wrist movement as a gesture.

公开(公告)号：US10997457B2

公开(公告)日：2021-05-04

申请号：US16616235

申请日：2019-10-16

Applicant: Google LLC

Inventor： Christoph Rhemann ,  Abhimitra Meka ,  Matthew Whalen ,  Jessica Lynn Busch ,  Sofien Bouaziz ,  Geoffrey Douglas Harvey ,  Andrea Tagliasacchi ,  Jonathan Taylor ,  Paul Debevec ,  Peter Joseph Denny ,  Sean Ryan Francesco Fanello ,  Graham Fyffe ,  Jason Angelo Dourgarian ,  Xueming Yu ,  Adarsh Prakash Murthy Kowdle ,  Julien Pascal Christophe Valentin ,  Peter Christopher Lincoln ,  Rohit Kumar Pandey ,  Christian Häne ,  Shahram Izadi

IPC: G06K9/46 ,  G06T15/50 ,  G06T15/20 ,  G06N3/08 ,  G06K9/62

Abstract: Methods, systems, and media for relighting images using predicted deep reflectance fields are provided. In some embodiments, the method comprises: identifying a group of training samples, wherein each training sample includes (i) a group of one-light-at-a-time (OLAT) images that have each been captured when one light of a plurality of lights arranged on a lighting structure has been activated, (ii) a group of spherical color gradient images that have each been captured when the plurality of lights arranged on the lighting structure have been activated to each emit a particular color, and (iii) a lighting direction, wherein each image in the group of OLAT images and each of the spherical color gradient images are an image of a subject, and wherein the lighting direction indicates a relative orientation of a light to the subject; training a convolutional neural network using the group of training samples, wherein training the convolutional neural network comprises: for each training iteration in a series of training iterations and for each training sample in the group of training samples: generating an output predicted image, wherein the output predicted image is a representation of the subject associated with the training sample with lighting from the lighting direction associated with the training sample; identifying a ground-truth OLAT image included in the group of OLAT images for the training sample that corresponds to the lighting direction for the training sample; calculating a loss that indicates a perceptual difference between the output predicted image and the identified ground-truth OLAT image; and updating parameters of the convolutional neural network based on the calculated loss; identifying a test sample that includes a second group of spherical color gradient images and a second lighting direction; and generating a relit image of the subject included in each of the second group of spherical color gradient images with lighting from the second lighting direction using the trained convolutional neural network.