-
公开(公告)号：US20210004979A1
公开(公告)日：2021-01-07
申请号：US16767401
申请日：2019-10-04
Applicant: Google LLC
Inventor： Jullien VALENTIN , Onur G. GULERYUZ , Mira LEUNG , Maksym DZITSIUK , Jose PASCOAL , Mirko SCHMIDT , Christoph RHEMANN , Neal WADHWA , Eric TURNER , Sameh KHAMIS , Adarsh Prakash Murthy KOWDLE , Ambrus CSASZAR , João Manuel Castro AFONSO , Jonathan T. BARRON , Michael SCHOENBERG , Ivan DRYANOVSKI , Vivek VERMA , Vladimir TANKOVICH , Shahram IZADI , Sean Ryan Francesco FANELLO , Konstantine Nicholas John TSOTSOS
Abstract: A handheld user device includes a monocular camera to capture a feed of images of a local scene and a processor to select, from the feed, a keyframe and perform, for a first image from the feed, stereo matching using the first image, the keyframe, and a relative pose based on a pose associated with the first image and a pose associated with the keyframe to generate a sparse disparity map representing disparities between the first image and the keyframe. The processor further is to determine a dense depth map from the disparity map using a bilateral solver algorithm, and process a viewfinder image generated from a second image of the feed with occlusion rendering based on the depth map to incorporate one or more virtual objects into the viewfinder image to generate an AR viewfinder image. Further, the processor is to provide the AR viewfinder image for display.
-
公开(公告)号：US20200099920A1
公开(公告)日：2020-03-26
申请号：US16580802
申请日：2019-09-24
Applicant: Google LLC
Inventor： Sameh KHAMIS , Yinda ZHANG , Christoph RHEMANN , Julien VALENTIN , Adarsh KOWDLE , Vladimir TANKOVICH , Michael SCHOENBERG , Shahram IZADI , Thomas FUNKHOUSER , Sean FANELLO
IPC: H04N13/271 , G06T7/90 , G06T7/521 , H04N5/33 , H04N13/239
Abstract: An electronic device estimates a depth map of an environment based on matching reduced-resolution stereo depth images captured by depth cameras to generate a coarse disparity (depth) map. The electronic device downsamples depth images captured by the depth cameras and matches sections of the reduced-resolution images to each other to generate a coarse depth map. The electronic device upsamples the coarse depth map to a higher resolution and refines the upsampled depth map to generate a high-resolution depth map to support location-based functionality.
-
公开(公告)号：US20200160109A1
公开(公告)日：2020-05-21
申请号：US16749626
申请日：2020-01-22
Applicant: Google LLC
Inventor： Sean Ryan FANELLO , Julien Pascal Christophe VALENTIN , Adarsh Prakash Murthy KOWDLE , Christoph RHEMANN , Vladimir TANKOVICH , Philip L. DAVIDSON , Shahram IZADI
IPC: G06K9/62
Abstract: Values of pixels in an image are mapped to a binary space using a first function that preserves characteristics of values of the pixels. Labels are iteratively assigned to the pixels in the image in parallel based on a second function. The label assigned to each pixel is determined based on values of a set of nearest-neighbor pixels. The first function is trained to map values of pixels in a set of training images to the binary space and the second function is trained to assign labels to the pixels in the set of training images. Considering only the nearest neighbors in the inference scheme results in a computational complexity that is independent of the size of the solution space and produces sufficient approximations of the true distribution when the solution for each pixel is most likely found in a small subset of the set of potential solutions.
-
公开(公告)号：US20180356883A1
公开(公告)日：2018-12-13
申请号：US16002595
申请日：2018-06-07
Applicant: Google LLC
Abstract: An electronic device estimates a pose of a face by fitting a generative face model mesh to a depth map based on vertices of the face model mesh that are estimated to be visible from the point of view of a depth camera. A face tracking module of the electronic device receives a depth image of a face from a depth camera and generates a depth map of the face based on the depth image. The face tracking module identifies a pose of the face by fitting a face model mesh to the pixels of a depth map that correspond to the vertices of the face model mesh that are estimated to be visible from the point of view of the depth camera.
-
公开(公告)号：US20180350088A1
公开(公告)日：2018-12-06
申请号：US15994471
申请日：2018-05-31
Applicant: Google LLC
Inventor： Mingsong DOU , Sean Ryan FANELLO , Adarsh Prakash Murthy KOWDLE , Christoph RHEMANN , Sameh KHAMIS , Philip L. DAVIDSON , Shahram IZADI , Vladimir Tankovich
CPC classification number: G06T7/75 , G06T2207/10016 , G06T2207/10024 , G06T2207/10028
Abstract: An electronic device estimates a pose of one or more subjects in an environment based on estimating a correspondence between a data volume containing a data mesh based on a current frame captured by a depth camera and a reference volume containing a plurality of fused prior data frames based on spectral embedding and performing bidirectional non-rigid matching between the reference volume and the current data frame to refine the correspondence so as to support location-based functionality. The electronic device predicts correspondences between the data volume and the reference volume based on spectral embedding. The correspondences provide constraints that accelerate the convergence between the data volume and the reference volume. By tracking changes between the current data mesh frame and the reference volume, the electronic device avoids tracking failures that can occur when relying solely on a previous data mesh frame.
-
Patent Agency Ranking
公开(公告)号：US20230419600A1
公开(公告)日：2023-12-28
申请号：US18251743
申请日：2020-11-05
Applicant: GOOGLE LLC
Inventor： Sean Ryan Francesco FANELLO , Abhi MEKA , Rohit Kumar PANDEY , Christian HAENE , Sergio Orts ESCOLANO , Christoph RHEMANN , Paul DEBEVEC , Sofien BOUAZIZ , Thabo BEELER , Ryan OVERBECK , Peter BARNUM , Daniel ERICKSON , Philip DAVIDSON , Yinda ZHANG , Jonathan TAYLOR , Chloe LeGENDRE , Shahram IZADI
CPC classification number: G06T15/506 , G06T7/55 , G06T15/04 , G06T15/20 , G06T7/60 , G06T2207/10048 , G06T2207/20084 , G06T2207/10152 , G06T2207/30196
Abstract: Example embodiments relate to techniques for volumetric performance capture with neural rendering. A technique may involve initially obtaining images that depict a subject from multiple viewpoints and under various lighting conditions using a light stage and depth data corresponding to the subject using infrared cameras. A neural network may extract features of the subject from the images based on the depth data and map the features into a texture space (e.g., the UV texture space). A neural renderer can be used to generate an output image depicting the subject from a target view such that illumination of the subject in the output image aligns with the target view. The neural render may resample the features of the subject from the texture space to an image space to generate the output image.
-
公开(公告)号：US20230209036A1
公开(公告)日：2023-06-29
申请号：US18111292
申请日：2023-02-17
Applicant: GOOGLE LLC
Inventor： Sameh KHAMIS , Yinda ZHANG , Christoph RHEMANN , Julien VALENTIN , Adarsh KOWDLE , Vladimir TANKOVICH , Michael SCHOENBERG , Shahram IZADI , Thomas FUNKHOUSER , Sean FANELLO
IPC: H04N13/271 , G06T7/90 , G06T7/521 , H04N5/33 , H04N13/239
CPC classification number: H04N13/271 , G06T7/90 , G06T7/521 , H04N5/33 , H04N13/239 , G06T2207/10024 , G06T2207/10028
Abstract: An electronic device estimates a depth map of an environment based on matching reduced-resolution stereo depth images captured by depth cameras to generate a coarse disparity (depth) map. The electronic device downsamples depth images captured by the depth cameras and matches sections of the reduced-resolution images to each other to generate a coarse depth map. The electronic device upsamples the coarse depth map to a higher resolution and refines the upsampled depth map to generate a high-resolution depth map to support location-based functionality.
-
公开(公告)号：US20180350105A1
公开(公告)日：2018-12-06
申请号：US15994563
申请日：2018-05-31
Applicant: Google LLC
Inventor： Jonathan James TAYLOR , Vladimir TANKOVICH , Danhang TANG , Cem KESKIN , Adarsh Prakash Murthy KOWDLE , Philip L. DAVIDSON , Shahram IZADI , David KIM
CPC classification number: G06T7/75 , G06K9/00382 , G06T7/149 , G06T7/162 , G06T7/194 , G06T7/251 , G06T2207/10016 , G06T2207/10028 , G06T2207/20076 , G06T2207/20081
Abstract: An electronic device estimates a pose of a hand by volumetrically deforming a signed distance field using a skinned tetrahedral mesh to locate a local minimum of an energy function, wherein the local minimum corresponds to the hand pose. The electronic device identifies a pose of the hand by fitting an implicit surface model of a hand to the pixels of a depth image that correspond to the hand. The electronic device uses a skinned tetrahedral mesh to warp space from a base pose to a deformed pose to define an articulated signed distance field from which the hand tracking module derives candidate poses of the hand. The electronic device then minimizes an energy function based on the distance of each corresponding pixel to identify the candidate pose that most closely approximates the pose of the hand.
-
公开(公告)号：US20220343525A1
公开(公告)日：2022-10-27
申请号：US17640114
申请日：2020-04-27
Applicant: GOOGLE LLC
Inventor： Rahul GARG , Neal WADHWA , Sean FANELLO , Christian HAENE , Yinda ZHANG , Sergio Orts ESCOLANO , Yael Pritch KNAAN , Marc LEVOY , Shahram IZADI
Abstract: Example implementations relate to joint depth prediction from dual cameras and dual pixels. An example method may involve obtaining a first set of depth information representing a scene from a first source and a second set of depth information representing the scene from a second source. The method may further involve determining, using a neural network, a joint depth map that conveys respective depths for elements in the scene. The neural network may determine the joint depth map based on a combination of the first set of depth information and the second set of depth information. In addition, the method may involve modifying an image representing the scene based on the joint depth map. For example, background portions of the image may be partially blurred based on the joint depth map.
-
公开(公告)号：US20180350087A1
公开(公告)日：2018-12-06
申请号：US15994509
申请日：2018-05-31
Applicant: Google LLC
Inventor： Adarsh Prakash Murthy KOWDLE , Vladimir TANKOVICH , Danhang TANG , Cem KESKIN , Jonathan James Taylor , Philip L. DAVIDSON , Shahram IZADI , Sean Ryan FANELLO , Julien Pascal Christophe VALENTIN , Christoph RHEMANN , Mingsong DOU , Sameh KHAMIS , David KIM
IPC: G06T7/593 , G06T7/32 , G06T7/33 , H04N13/271
CPC classification number: G06T7/521 , G06T7/593 , G06T2207/10021 , G06T2207/10024 , G06T2207/10152 , G06T2207/20072 , G06T2207/20076 , G06T2207/20081
Abstract: An electronic device estimates a depth map of an environment based on stereo depth images captured by depth cameras having exposure times that are offset from each other in conjunction with illuminators pulsing illumination patterns into the environment. A processor of the electronic device matches small sections of the depth images from the cameras to each other and to corresponding patches of immediately preceding depth images (e.g., a spatio-temporal image patch “cube”). The processor computes a matching cost for each spatio-temporal image patch cube by converting each spatio-temporal image patch into binary codes and defining a cost function between two stereo image patches as the difference between the binary codes. The processor minimizes the matching cost to generate a disparity map, and optimizes the disparity map by rejecting outliers using a decision tree with learned pixel offsets and refining subpixels to generate a depth map of the environment.
-
-
-
-
-
-
-
-
-
Search Results
11
records
(took 0.012 seconds)
