-
11.发明申请公开(公告)号：US20230040793A1
公开(公告)日：2023-02-09
申请号：US17870462
申请日：2022-07-21
Applicant: Google LLC
Inventor： Avneesh Sud , Andrea Tagliasacchi , Ben Usman
Abstract: Example systems perform complex optimization tasks with improved efficiency via neural meta-optimization of experts. In particular, provided is a machine learning framework in which a meta-optimization neural network can learn to fuse a collection of experts to provide a predicted solution. Specifically, the meta-optimization neural network can learn to predict the output of a complex optimization process which optimizes over outputs from the collection of experts to produce an optimized output. In such fashion, the meta-optimization neural network can, after training, be used in place of the complex optimization process to produce a synthesized solution from the experts, leading to orders of magnitude faster and computationally more efficient prediction or problem solution.
-
公开(公告)号：US11328486B2
公开(公告)日：2022-05-10
申请号：US16861530
申请日：2020-04-29
Applicant: Google LLC
Inventor： Anastasia Tkach , Ricardo Martin Brualla , Shahram Izadi , Shuoran Yang , Cem Keskin , Sean Ryan Francesco Fanello , Philip Davidson , Jonathan Taylor , Rohit Pandey , Andrea Tagliasacchi , Pavlo Pidlypenskyi
Abstract: A method includes receiving a first image including color data and depth data, determining a viewpoint associated with an augmented reality (AR) and/or virtual reality (VR) display displaying a second image, receiving at least one calibration image including an object in the first image, the object being in a different pose as compared to a pose of the object in the first image, and generating the second image based on the first image, the viewpoint and the at least one calibration image.
-
Patent Agency Ranking
13.发明申请公开(公告)号：US20200372284A1
公开(公告)日：2020-11-26
申请号：US16616235
申请日：2019-10-16
Applicant: Google LLC
Inventor： Christoph Rhemann , Abhimitra Meka , Matthew Whalen , Jessica Lynn Busch , Sofien Bouaziz , Geoffrey Douglas Harvey , Andrea Tagliasacchi , Jonathan Taylor , Paul Debevec , Peter Joseph Denny , Sean Ryan Francesco Fanello , Graham Fyffe , Jason Angelo Dourgarian , Xueming Yu , Adarsh Prakash Murthy Kowdle , Julien Pascal Christophe Valentin , Peter Christopher Lincoln , Rohit Kumar Pandey , Christian Häne , Shahram Izadi
Abstract: Methods, systems, and media for relighting images using predicted deep reflectance fields are provided. In some embodiments, the method comprises: identifying a group of training samples, wherein each training sample includes (i) a group of one-light-at-a-time (OLAT) images that have each been captured when one light of a plurality of lights arranged on a lighting structure has been activated, (ii) a group of spherical color gradient images that have each been captured when the plurality of lights arranged on the lighting structure have been activated to each emit a particular color, and (iii) a lighting direction, wherein each image in the group of OLAT images and each of the spherical color gradient images are an image of a subject, and wherein the lighting direction indicates a relative orientation of a light to the subject; training a convolutional neural network using the group of training samples, wherein training the convolutional neural network comprises: for each training iteration in a series of training iterations and for each training sample in the group of training samples: generating an output predicted image, wherein the output predicted image is a representation of the subject associated with the training sample with lighting from the lighting direction associated with the training sample; identifying a ground-truth OLAT image included in the group of OLAT images for the training sample that corresponds to the lighting direction for the training sample; calculating a loss that indicates a perceptual difference between the output predicted image and the identified ground-truth OLAT image; and updating parameters of the convolutional neural network based on the calculated loss; identifying a test sample that includes a second group of spherical color gradient images and a second lighting direction; and generating a relit image of the subject included in each of the second group of spherical color gradient images with lighting from the second lighting direction using the trained convolutional neural network.
-
14.发明申请公开(公告)号：US20240420413A1
公开(公告)日：2024-12-19
申请号：US18813675
申请日：2024-08-23
Applicant: Google LLC
Inventor： Konstantinos Rematas , Thomas Allen Funkhouser , Vittorio Carlo Ferrari , Andrew Huaming Liu , Andrea Tagliasacchi , Pratul Preeti Srinivasan , Jonathan Tilton Barron
Abstract: Systems and methods for view synthesis and three-dimensional reconstruction can learn an environment by utilizing a plurality of images of an environment and depth data. The use of depth data can be helpful when the quantity of images and different angles may be limited. For example, large outdoor environments can be difficult to learn due to the size, the varying image exposures, and the limited variance in view direction changes. The systems and methods can leverage a plurality of panoramic images and corresponding lidar data to accurately learn a large outdoor environment to then generate view synthesis outputs and three-dimensional reconstruction outputs. Training may include the use of an exposure correction network to address lighting exposure differences between training images.
-
15.发明申请公开(公告)号：US20240371081A1
公开(公告)日：2024-11-07
申请号：US18688278
申请日：2022-04-13
Applicant: Google LLC
Inventor： Mark Jeffrey Matthews , Daniel Jonathan Rebain , Dmitry Lagun , Andrea Tagliasacchi
IPC: G06T15/20 , G06T15/08 , G06V10/774 , G06V40/16
Abstract: Systems and methods for learning spaces of three-dimensional shape and appearance from datasets of single-view images can be utilized for generating view renderings of a variety of different objects and/or scenes. The systems and methods can be able to learn effectively from unstructured. “in-the-wild” data, without incurring the high cost of a full-image discriminator, and while avoiding problems such as mode-dropping that are inherent to adversarial methods.
-
公开(公告)号：US20240096001A1
公开(公告)日：2024-03-21
申请号：US18013983
申请日：2022-11-15
Applicant: Google LLC
Inventor： Seyed Mohammad Mehdi Sajjadi , Henning Meyer , Etienne François Régis Pot , Urs Michael Bergmann , Klaus Greff , Noha Radwan , Suhani Deepak-Ranu Vora , Mario Lu¢i¢ , Daniel Christopher Duckworth , Thomas Allen Funkhouser , Andrea Tagliasacchi
Abstract: Provided are machine learning models that generate geometry-free neural scene representations through efficient object-centric novel-view synthesis. In particular, one example aspect of the present disclosure provides a novel framework in which an encoder model (e.g., an encoder transformer network) processes one or more RGB images (with or without pose) to produce a fully latent scene representation that can be passed to a decoder model (e.g., a decoder transformer network). Given one or more target poses, the decoder model can synthesize images in a single forward pass. In some example implementations, because transformers are used rather than convolutional or MLP networks, the encoder can learn an attention model that extracts enough 3D information about a scene from a small set of images to render novel views with correct projections, parallax, occlusions, and even semantics, without explicit geometry.
-
17.发明授权公开(公告)号：US10997457B2
公开(公告)日：2021-05-04
申请号：US16616235
申请日：2019-10-16
Applicant: Google LLC
Inventor： Christoph Rhemann , Abhimitra Meka , Matthew Whalen , Jessica Lynn Busch , Sofien Bouaziz , Geoffrey Douglas Harvey , Andrea Tagliasacchi , Jonathan Taylor , Paul Debevec , Peter Joseph Denny , Sean Ryan Francesco Fanello , Graham Fyffe , Jason Angelo Dourgarian , Xueming Yu , Adarsh Prakash Murthy Kowdle , Julien Pascal Christophe Valentin , Peter Christopher Lincoln , Rohit Kumar Pandey , Christian Häne , Shahram Izadi
Abstract: Methods, systems, and media for relighting images using predicted deep reflectance fields are provided. In some embodiments, the method comprises: identifying a group of training samples, wherein each training sample includes (i) a group of one-light-at-a-time (OLAT) images that have each been captured when one light of a plurality of lights arranged on a lighting structure has been activated, (ii) a group of spherical color gradient images that have each been captured when the plurality of lights arranged on the lighting structure have been activated to each emit a particular color, and (iii) a lighting direction, wherein each image in the group of OLAT images and each of the spherical color gradient images are an image of a subject, and wherein the lighting direction indicates a relative orientation of a light to the subject; training a convolutional neural network using the group of training samples, wherein training the convolutional neural network comprises: for each training iteration in a series of training iterations and for each training sample in the group of training samples: generating an output predicted image, wherein the output predicted image is a representation of the subject associated with the training sample with lighting from the lighting direction associated with the training sample; identifying a ground-truth OLAT image included in the group of OLAT images for the training sample that corresponds to the lighting direction for the training sample; calculating a loss that indicates a perceptual difference between the output predicted image and the identified ground-truth OLAT image; and updating parameters of the convolutional neural network based on the calculated loss; identifying a test sample that includes a second group of spherical color gradient images and a second lighting direction; and generating a relit image of the subject included in each of the second group of spherical color gradient images with lighting from the second lighting direction using the trained convolutional neural network.
-
-
-
-
-
-
Search Results
17
records
(took 0.014 seconds)
