公开(公告)号：US20230154051A1

公开(公告)日：2023-05-18

申请号：US17919460

申请日：2020-04-17

Applicant: Google LLC

Inventor： Danhang Tang ,  Saurabh Singh ,  Cem Keskin ,  Phillip Andrew Chou ,  Christian Haene ,  Mingsong Dou ,  Sean Ryan Francesco Fanello ,  Jonathan Taylor ,  Andrea Tagliasacchi ,  Philip Lindsley Davidson ,  Yinda Zhang ,  Onur Gonen Guleryuz ,  Shahram Izadi ,  Sofien Bouaziz

IPC: G06T9/00

CPC classification number: G06T9/001 ,  G06T9/002

Abstract: Systems and methods are directed to encoding and/or decoding of the textures/geometry of a three-dimensional volumetric representation. An encoding computing system can obtain voxel blocks from a three-dimensional volumetric representation of an object. The encoding computing system can encode voxel blocks with a machine-learned voxel encoding model to obtain encoded voxel blocks. The encoding computing system can decode the encoded voxel blocks with a machine-learned voxel decoding model to obtain reconstructed voxel blocks. The encoding computing system can generate a reconstructed mesh representation of the object based at least in part on the one or more reconstructed voxel blocks. The encoding computing system can encode textures associated with the voxel blocks according to an encoding scheme and based at least in part on the reconstructed mesh representation of the object to obtain encoded textures.

公开(公告)号：US20210264632A1

公开(公告)日：2021-08-26

申请号：US17249095

申请日：2021-02-19

Applicant: GOOGLE LLC

Inventor： Vladimir Tankovich ,  Christian Haene ,  Sean Rayn Francesco Fanello ,  Yinda Zhang ,  Shahram Izadi ,  Sofien Bouaziz ,  Adarsh Prakash Murthy Kowdle ,  Sameh Khamis

IPC: G06T7/593 ,  G06T3/00 ,  G06T3/40 ,  G06T5/30 ,  H04N13/20

Abstract: According to an aspect, a real-time active stereo system includes a capture system configured to capture stereo data, where the stereo data includes a first input image and a second input image, and a depth sensing computing system configured to predict a depth map. The depth sensing computing system includes a feature extractor configured to extract features from the first and second images at a plurality of resolutions, an initialization engine configured to generate a plurality of depth estimations, where each of the plurality of depth estimations corresponds to a different resolution, and a propagation engine configured to iteratively refine the plurality of depth estimations based on image warping and spatial propagation.

公开(公告)号：US20240290025A1

公开(公告)日：2024-08-29

申请号：US18588948

申请日：2024-02-27

Applicant: GOOGLE LLC

Inventor： Yinda Zhang ,  Sean Ryan Francesco Fanello ,  Ziqian Bai ,  Feitong Tan ,  Zeng Huang ,  Kripasindhu Sarkar ,  Danhang Tang ,  Di Qiu ,  Abhimitra Meka ,  Ruofei Du ,  Mingsong Dou ,  Sergio Orts Escolano ,  Rohit Kumar Pandey ,  Thabo Beeler

IPC: G06T13/40 ,  G06T7/90 ,  G06T17/20 ,  G06V10/44

CPC classification number: G06T13/40 ,  G06T7/90 ,  G06T17/20 ,  G06V10/44 ,  G06T2207/10024 ,  G06T2207/20084

Abstract: A method comprises receiving a first sequence of images of a portion of a user, the first sequence of images being monocular images; generating an avatar based on the first sequence of images, the avatar being based on a model including a feature vector associated with a vertex; receiving a second sequence of images of the portion of the user; and based on the second sequence of images, modifying the avatar with a displacement of the vertex to represent a gesture of the avatar.

公开(公告)号：US20240212325A1

公开(公告)日：2024-06-27

申请号：US18596822

申请日：2024-03-06

Applicant: Google LLC

Inventor： Yinda Zhang ,  Feitong Tan ,  Danhang Tang ,  Mingsong Dou ,  Kaiwen Guo ,  Sean Ryan Francesco Fanello ,  Sofien Bouaziz ,  Cem Keskin ,  Ruofei Du ,  Rohit Kumar Pandey ,  Deqing Sun

IPC: G06V10/771 ,  G06T7/70 ,  G06T17/00 ,  G06V10/44 ,  G06V10/75

CPC classification number: G06V10/771 ,  G06T7/70 ,  G06T17/00 ,  G06V10/44 ,  G06V10/751 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Systems and methods for training models to predict dense correspondences across images such as human images. A model may be trained using synthetic training data created from one or more 3D computer models of a subject. In addition, one or more geodesic distances derived from the surfaces of one or more of the 3D models may be used to generate one or more loss values, which may in turn be used in modifying the model's parameters during training.

公开(公告)号：US20240212184A1

公开(公告)日：2024-06-27

申请号：US18555059

申请日：2021-04-30

Applicant: Google LLC

Inventor： Ruofei Du ,  David Li ,  Danhang Tang ,  Yinda Zhang

IPC: G06T7/55 ,  G06T5/77 ,  G06T7/181 ,  G06T15/00 ,  G06T17/20

CPC classification number: G06T7/55 ,  G06T5/77 ,  G06T7/181 ,  G06T15/00 ,  G06T17/20 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221

Abstract: A method including predicting a stereo depth associated with a first panoramic image and a second panoramic image, the first panoramic image and the second panoramic image being captured with a time interlude between the capture of the first panoramic image and the second panoramic image, generating a first mesh representation based on the first panoramic image and a stereo depth corresponding to the first panoramic image, generating a second mesh representation based on the second panoramic image and a stereo depth corresponding to the second panoramic image, and synthesizing a third panoramic image based on fusing the first mesh representation with the second mesh representation.

公开(公告)号：US11810313B2

公开(公告)日：2023-11-07

申请号：US17249095

申请日：2021-02-19

Applicant: GOOGLE LLC

Inventor： Vladimir Tankovich ,  Christian Haene ,  Sean Ryan Francesco Fanello ,  Yinda Zhang ,  Shahram Izadi ,  Sofien Bouaziz ,  Adarsh Prakash Murthy Kowdle ,  Sameh Khamis

IPC: G06T7/593 ,  H04N13/20 ,  G06T3/00 ,  G06T3/40 ,  G06T5/30 ,  H04N13/00

CPC classification number: G06T7/593 ,  G06T3/0093 ,  G06T3/40 ,  G06T5/30 ,  H04N13/20 ,  G06T2207/20016 ,  G06T2207/20084 ,  H04N2013/0081

Abstract: According to an aspect, a real-time active stereo system includes a capture system configured to capture stereo data, where the stereo data includes a first input image and a second input image, and a depth sensing computing system configured to predict a depth map. The depth sensing computing system includes a feature extractor configured to extract features from the first and second images at a plurality of resolutions, an initialization engine configured to generate a plurality of depth estimations, where each of the plurality of depth estimations corresponds to a different resolution, and a propagation engine configured to iteratively refine the plurality of depth estimations based on image warping and spatial propagation.

公开(公告)号：US20220405500A1

公开(公告)日：2022-12-22

申请号：US17304419

申请日：2021-06-21

Applicant: Google LLC

Inventor： Mayank Bhargava ,  Idris Syed Aleem ,  Yinda Zhang ,  Sushant Umesh Kulkarni ,  Rees Anwyl Simmons ,  Ahmed Gawish

IPC: G06K9/00 ,  G06T7/73 ,  G06K9/32 ,  G06K9/62 ,  G06T17/00 ,  G06T7/50 ,  G06T19/20 ,  G06T7/246 ,  G02C7/02

Abstract: A computer-implemented method includes receiving a two-dimensional (2-D) side view face image of a person, identifying a bounded portion or area of the 2-D side view face image of the person as an ear region-of-interest (ROI) area showing at least a portion of an ear of the person, and processing the identified ear ROI area of the 2-D side view face image, pixel-by-pixel, through a trained fully convolutional neural network model (FCNN model) to predict a 2-D ear saddle point (ESP) location for the ear shown in the ear ROI area. The FCNN model has an image segmentation architecture.

公开(公告)号：US20240129437A1

公开(公告)日：2024-04-18

申请号：US18047420

申请日：2022-10-18

Applicant: Google LLC

Inventor： Yinda Zhang ,  Ruofei Du

IPC: H04N7/15 ,  G06F3/16 ,  G06T13/00

CPC classification number: H04N7/157 ,  G06F3/167 ,  G06T13/00

Abstract: A method can include selecting, from at least a first avatar and a second avatar based on at least one attribute of a calendar event associated with a user, a session avatar, the first avatar being based on a first set of images of a user wearing a first outfit and the second avatar being based on a second set of images of the user wearing a second outfit, and presenting the session avatar during a videoconference, the presentation of the session avatar changing based on audio input received from the user during the videoconference.

公开(公告)号：US20240062046A1

公开(公告)日：2024-02-22

申请号：US18270685

申请日：2021-03-31

Applicant: Google LLC

Inventor： Ruofei Du ,  Yinda Zhang ,  Weihao Zeng

IPC: G06N3/0464 ,  G06V10/82 ,  G06V10/42 ,  G06V10/44 ,  G06N3/084

CPC classification number: G06N3/0464 ,  G06V10/82 ,  G06V10/42 ,  G06V10/44 ,  G06N3/084

Abstract: A system including a computer vision model configured to perform a machine learning task is described. The computer vision model includes multiple wrapped convolutional layers, in which each wrapped convolutional layer includes a respective convolutional layer configured to receive, for each time step of multiple time steps, a layer input and to process the layer input to generate an initial output for the current time step, and a respective note-taking module configured to receive the initial output and to process the initial output to generate a feature vector for the current time step, the feature vector representing local information of the wrapped convolutional layer. The model includes a summarization module configured to receive the feature vectors and to process the feature vectors to generate a revision vector for the current time step, the revision vector representing global information of the plurality of wrapped convolutional layers.

公开(公告)号：US20240020915A1

公开(公告)日：2024-01-18

申请号：US18353213

申请日：2023-07-17

Applicant: Google LLC

Inventor： Yinda Zhang ,  Feitong Tan ,  Sean Ryan Francesco Fanello ,  Abhimitra Meka ,  Sergio Orts Escolano ,  Danhang Tang ,  Rohit Kumar Pandey ,  Jonathan James Taylor

IPC: G06T15/80 ,  G06T15/08

CPC classification number: G06T15/80 ,  G06T15/08

Abstract: Techniques include introducing a neural generator configured to produce novel faces that can be rendered at free camera viewpoints (e.g., at any angle with respect to the camera) and relit under an arbitrary high dynamic range (HDR) light map. A neural implicit intrinsic field takes a randomly sampled latent vector as input and produces as output per-point albedo, volume density, and reflectance properties for any queried 3D location. These outputs are aggregated via a volumetric rendering to produce low resolution albedo, diffuse shading, specular shading, and neural feature maps. The low resolution maps are then upsampled to produce high resolution maps and input into a neural renderer to produce relit images.