公开(公告)号：US20230134779A1

公开(公告)日：2023-05-04

申请号：US18050058

申请日：2022-10-27

Applicant: Google LLC

Inventor： Bernhard Reinert ,  Sebastian Sylvan ,  Hugues Hoppe ,  Grigory Javadyan ,  Egor Yusov ,  David Chu ,  Kelvin Ritland

IPC: G06T17/20 ,  G06T7/50 ,  G06V10/74

Abstract: A method includes obtaining a first frame of image data including a plurality of pixels. Each pixel of the plurality of pixels is associated with a respective color value and a respective depth value. The first frame renders a scene from a first point of view. The method includes generating a three-dimensional (3D) polygon mesh using the plurality of pixels and the respective depth values. The 3D polygon mesh includes a plurality of portions. Each respective portion defines a respective plurality of vertices defining a respective mesh density representative of a density of the respective plurality of vertices. Each vertex is associated with a corresponding pixel of the first frame of image data. The method includes generating a second frame of image data via reprojection using the generated 3D polygon mesh. The second frame of image data has a second point of view different from the first point of view.

公开(公告)号：US10757410B1

公开(公告)日：2020-08-25

申请号：US16523247

申请日：2019-07-26

Applicant: GOOGLE LLC

Inventor： Hugues Hoppe ,  True Price

IPC: H04N19/126 ,  H04N19/176

Abstract: Techniques of compressing color video images include computing a delta quantization parameter (ΔQP) for the color images based on a similarity between the depth image surface normal and the view direction associated with a color image. For example, upon receiving a frame having an image with multiple color and depth images, a computer finds a depth image that is closest in orientation to a color image. For each pixel of that depth image, the computer generates a blend weight based on an orientation of a normal to a position of the depth image and the viewpoints from which the plurality of color images were captured. The computer then generates a value of ΔQP based on the blend weight and determines a macroblock of color image corresponding to the position, the macroblock being associated with the value of ΔQP for the pixel.

公开(公告)号：US10818071B1

公开(公告)日：2020-10-27

申请号：US16523702

申请日：2019-07-26

Applicant: Google LLC

Inventor： Hugues Hoppe ,  Ricardo Martin Brualla ,  Harris Nover

IPC: G06T15/06 ,  G06T7/70 ,  G06T7/55 ,  G06T15/20 ,  H04N13/111

Abstract: Techniques of rendering images includes generating signed distance values (SDVs) along a ray from a specified viewpoint in terms of projected distances along that ray from given depth images. For each pixel in an image of from the perspective of the specified viewpoint, a ray is traced into the three-dimensional scene represented by the image. An iterative step is performed along the ray, obtaining in each iteration a three-dimensional world-space point p. The result is the signed distance sj as measured from depth view Dj. If the absolute value of the signed distance sj is greater than some truncation threshold parameter, the signed distance sj is replaced by a special undefined value. The defined signed-distance values are aggregated to obtain an overall signed distance s. Finally, the roots or zero set (isosurface) of the signed distance field is determined.

公开(公告)号：US11288857B2

公开(公告)日：2022-03-29

申请号：US16837612

申请日：2020-04-01

Applicant: Google LLC

Inventor： Moustafa Meshry ,  Ricardo Martin Brualla ,  Sameh Khamis ,  Daniel Goldman ,  Hugues Hoppe ,  Noah Snavely ,  Rohit Pandey

IPC: G06T15/00 ,  G06T15/20 ,  G06T7/174 ,  G06N3/08

Abstract: According to an aspect, a method for neural rerendering includes obtaining a three-dimensional (3D) model representing a scene of a physical space, where the 3D model is constructed from a collection of input images, rendering an image data buffer from the 3D model according to a viewpoint, where the image data buffer represents a reconstructed image from the 3D model, receiving, by a neural rerendering network, the image data buffer, receiving, by the neural rerendering network, an appearance code representing an appearance condition, and transforming, by the neural rerendering network, the image data buffer into a rerendered image with the viewpoint of the image data buffer and the appearance condition specified by the appearance code.

公开(公告)号：US20200320777A1

公开(公告)日：2020-10-08

申请号：US16837612

申请日：2020-04-01

Applicant: Google LLC

Inventor： Moustafa Meshry ,  Ricardo Martin Brualla ,  Sameh Khamis ,  Daniel Goldman ,  Hugues Hoppe ,  Noah Snavely ,  Rohit Pandey

IPC: G06T15/20 ,  G06T7/174 ,  G06N3/08

Abstract: According to an aspect, a method for neural rerendering includes obtaining a three-dimensional (3D) model representing a scene of a physical space, where the 3D model is constructed from a collection of input images, rendering an image data buffer from the 3D model according to a viewpoint, where the image data buffer represents a reconstructed image from the 3D model, receiving, by a neural rerendering network, the image data buffer, receiving, by the neural rerendering network, an appearance code representing an appearance condition, and transforming, by the neural rerendering network, the image data buffer into a rerendered image with the viewpoint of the image data buffer and the appearance condition specified by the appearance code.