-
公开(公告)号:US20230154051A1
公开(公告)日:2023-05-18
申请号:US17919460
申请日:2020-04-17
Applicant: Google LLC
Inventor: Danhang Tang , Saurabh Singh , Cem Keskin , Phillip Andrew Chou , Christian Haene , Mingsong Dou , Sean Ryan Francesco Fanello , Jonathan Taylor , Andrea Tagliasacchi , Philip Lindsley Davidson , Yinda Zhang , Onur Gonen Guleryuz , Shahram Izadi , Sofien Bouaziz
IPC: G06T9/00
Abstract: Systems and methods are directed to encoding and/or decoding of the textures/geometry of a three-dimensional volumetric representation. An encoding computing system can obtain voxel blocks from a three-dimensional volumetric representation of an object. The encoding computing system can encode voxel blocks with a machine-learned voxel encoding model to obtain encoded voxel blocks. The encoding computing system can decode the encoded voxel blocks with a machine-learned voxel decoding model to obtain reconstructed voxel blocks. The encoding computing system can generate a reconstructed mesh representation of the object based at least in part on the one or more reconstructed voxel blocks. The encoding computing system can encode textures associated with the voxel blocks according to an encoding scheme and based at least in part on the reconstructed mesh representation of the object to obtain encoded textures.
-
公开(公告)号:US11592908B2
公开(公告)日:2023-02-28
申请号:US17249966
申请日:2021-03-19
Applicant: GOOGLE LLC
Inventor: Dongeek Shin , Shahram Izadi , David Kim , Sofien Bouaziz , Steven Benjamin Goldberg , Ivan Poupyrev , Shwetak N. Patel
Abstract: Techniques of identifying gestures include detecting and classifying inner-wrist muscle motions at a user's wrist using micron-resolution radar sensors. For example, a user of an AR system may wear a band around their wrist. When the user makes a gesture to manipulate a virtual object in the AR system as seen in a head-mounted display (HMD), muscles and ligaments in the user's wrist make small movements on the order of 1-3 mm. The band contains a small radar device that has a transmitter and a number of receivers (e.g., three) of electromagnetic (EM) radiation on a chip (e.g., a Soli chip. This radiation reflects off the wrist muscles and ligaments and is received by the receivers on the chip in the band. The received reflected signal, or signal samples, are then sent to processing circuitry for classification to identify the wrist movement as a gesture.
-
公开(公告)号:US20220350997A1
公开(公告)日:2022-11-03
申请号:US17302291
申请日:2021-04-29
Applicant: Google LLC
Inventor: Qinge Wu , Grant Yoshida , Catherine Boulanger , Erik Hubert Dolly Goossens , Cem Keskin , Sofien Bouaziz , Jonathan James Taylor , Nidhi Rathi , Seth Raphael
Abstract: A head-mounted device (HMD) can be configured to determine a request for recognizing at least one content item included within content framed within a display of the HMD. The HMD can be configured to initiate a head-tracking process that maintains a coordinate system with respect to the content, and a pointer-tracking process that tracks a pointer that is visible together with the content within the display. The HMD can be configured to capture a first image of the content and a second image of the content, the second image including the pointer. The HMD can be configured to map a location of the pointer within the second image to a corresponding image location within the first image, using the coordinate system, and provide the at least one content item from the corresponding image location.
-
公开(公告)号:US20210264632A1
公开(公告)日:2021-08-26
申请号:US17249095
申请日:2021-02-19
Applicant: GOOGLE LLC
Inventor: Vladimir Tankovich , Christian Haene , Sean Rayn Francesco Fanello , Yinda Zhang , Shahram Izadi , Sofien Bouaziz , Adarsh Prakash Murthy Kowdle , Sameh Khamis
Abstract: According to an aspect, a real-time active stereo system includes a capture system configured to capture stereo data, where the stereo data includes a first input image and a second input image, and a depth sensing computing system configured to predict a depth map. The depth sensing computing system includes a feature extractor configured to extract features from the first and second images at a plurality of resolutions, an initialization engine configured to generate a plurality of depth estimations, where each of the plurality of depth estimations corresponds to a different resolution, and a propagation engine configured to iteratively refine the plurality of depth estimations based on image warping and spatial propagation.
-
25.
公开(公告)号:US20200372284A1
公开(公告)日:2020-11-26
申请号:US16616235
申请日:2019-10-16
Applicant: Google LLC
Inventor: Christoph Rhemann , Abhimitra Meka , Matthew Whalen , Jessica Lynn Busch , Sofien Bouaziz , Geoffrey Douglas Harvey , Andrea Tagliasacchi , Jonathan Taylor , Paul Debevec , Peter Joseph Denny , Sean Ryan Francesco Fanello , Graham Fyffe , Jason Angelo Dourgarian , Xueming Yu , Adarsh Prakash Murthy Kowdle , Julien Pascal Christophe Valentin , Peter Christopher Lincoln , Rohit Kumar Pandey , Christian Häne , Shahram Izadi
Abstract: Methods, systems, and media for relighting images using predicted deep reflectance fields are provided. In some embodiments, the method comprises: identifying a group of training samples, wherein each training sample includes (i) a group of one-light-at-a-time (OLAT) images that have each been captured when one light of a plurality of lights arranged on a lighting structure has been activated, (ii) a group of spherical color gradient images that have each been captured when the plurality of lights arranged on the lighting structure have been activated to each emit a particular color, and (iii) a lighting direction, wherein each image in the group of OLAT images and each of the spherical color gradient images are an image of a subject, and wherein the lighting direction indicates a relative orientation of a light to the subject; training a convolutional neural network using the group of training samples, wherein training the convolutional neural network comprises: for each training iteration in a series of training iterations and for each training sample in the group of training samples: generating an output predicted image, wherein the output predicted image is a representation of the subject associated with the training sample with lighting from the lighting direction associated with the training sample; identifying a ground-truth OLAT image included in the group of OLAT images for the training sample that corresponds to the lighting direction for the training sample; calculating a loss that indicates a perceptual difference between the output predicted image and the identified ground-truth OLAT image; and updating parameters of the convolutional neural network based on the calculated loss; identifying a test sample that includes a second group of spherical color gradient images and a second lighting direction; and generating a relit image of the subject included in each of the second group of spherical color gradient images with lighting from the second lighting direction using the trained convolutional neural network.
-
公开(公告)号:US20240303908A1
公开(公告)日:2024-09-12
申请号:US18547628
申请日:2021-04-30
Applicant: GOOGLE LLC
Inventor: Yinda Zhang , Danhang Tang , Ruofei Du , Zhang Chen , Kyle Genova , Sofien Bouaziz , Thomas Allen Funkhouser , Sean Ryan Francesco Fanello , Christian Haene
Abstract: A method including generating a first vector based on a first grid and a three-dimensional (3D) position associated with a first implicit representation (IR) of a 3D object, generating at least one second vector based on at least one second grid and an upsampled first grid, decoding the first vector to generate a second IR of the 3D object, decoding the at least one second vector to generate at least one third IR of the 3D object, generating a composite IR of the 3D object based on the second IR of the 3D object and the at least one third IR of the 3D object, and generating a reconstructed volume representing the 3D object based on the composite IR of the 3D object.
-
公开(公告)号:US20240212106A1
公开(公告)日:2024-06-27
申请号:US18554960
申请日:2021-04-28
Applicant: Google LLC
Inventor: Chloe LeGendre , Paul Debevec , Sean Ryan Francesco Fanello , Rohit Kumar Pandey , Sergio Orts Escolano , Christian Haene , Sofien Bouaziz
CPC classification number: G06T5/50 , G06T7/194 , G06V10/56 , G06V10/60 , H04N5/272 , G06T2207/20221
Abstract: Apparatus and methods related to applying lighting models to images are provided. An example method includes receiving, via a computing device, an image comprising a subject. The method further includes relighting, via a neural network, a foreground of the image to maintain a consistent lighting of the foreground with a target illumination. The relighting is based on a per-pixel light representation indicative of a surface geometry of the foreground. The light representation includes a specular component, and a diffuse component, of surface reflection. The method additionally includes predicting, via the neural network, an output image comprising the subject in the relit foreground. One or more neural networks can be trained to perform one or more of the aforementioned aspects.
-
28.
公开(公告)号:US11954899B2
公开(公告)日:2024-04-09
申请号:US18274371
申请日:2021-03-11
Applicant: Google LLC
Inventor: Yinda Zhang , Feitong Tan , Danhang Tang , Mingsong Dou , Kaiwen Guo , Sean Ryan Francesco Fanello , Sofien Bouaziz , Cem Keskin , Ruofei Du , Rohit Kumar Pandey , Deqing Sun
IPC: G06V10/771 , G06T7/70 , G06T17/00 , G06V10/44 , G06V10/75
CPC classification number: G06V10/771 , G06T7/70 , G06T17/00 , G06V10/44 , G06V10/751 , G06T2207/20081 , G06T2207/20084
Abstract: Systems and methods for training models to predict dense correspondences across images such as human images. A model may be trained using synthetic training data created from one or more 3D computer models of a subject. In addition, one or more geodesic distances derived from the surfaces of one or more of the 3D models may be used to generate one or more loss values, which may in turn be used in modifying the model's parameters during training.
-
29.
公开(公告)号:US20240046618A1
公开(公告)日:2024-02-08
申请号:US18274371
申请日:2021-03-11
Applicant: Google LLC
Inventor: Yinda Zhang , Feitong Tan , Danhang Tang , Mingsong Dou , Kaiwen Guo , Sean Ryan Francesco Fanello , Sofien Bouaziz , Cem Keskin , Ruofei Du , Rohit Kumar Pandey , Deqing Sun
IPC: G06V10/771 , G06T17/00 , G06T7/70 , G06V10/44 , G06V10/75
CPC classification number: G06V10/771 , G06T17/00 , G06T7/70 , G06V10/44 , G06V10/751 , G06T2207/20081 , G06T2207/20084
Abstract: Systems and methods for training models to predict dense correspondences across images such as human images. A model may be trained using synthetic training data created from one or more 3D computer models of a subject. In addition, one or more geodesic distances derived from the surfaces of one or more of the 3D models may be used to generate one or more loss values, which may in turn be used in modifying the model's parameters during training.
-
公开(公告)号:US20240005590A1
公开(公告)日:2024-01-04
申请号:US18251995
申请日:2021-01-14
Applicant: GOOGLE LLC
Inventor: Ricardo Martin Brualla , Keunhong Park , Utkarsh Sinha , Sofien Bouaziz , Daniel Goldman , Jonathan Tilton Barron , Steven Maxwell Seitz
Abstract: Techniques of image synthesis using a neural radiance field (NeRF) includes generating a deformation model of movement experienced by a subject in a non-rigidly deforming scene. For example, when an image synthesis system uses NeRFs, the system takes as input multiple poses of subjects for training data. In contrast to conventional NeRFs, the technical solution first expresses the positions of the subjects from various perspectives in an observation frame. The technical solution then involves deriving a deformation model, i.e., a mapping between the observation frame and a canonical frame in which the subject's movements are taken into account. This mapping is accomplished using latent deformation codes for each pose that are determined using a multilayer perceptron (MLP). A NeRF is then derived from positions and casted ray directions in the canonical frame using another MLP. New poses for the subject may then be derived using the NeRF.
-
-
-
-
-
-
-
-
-