Adaptive Mesh Reprojection for Low Latency 6DOF Rendering

    公开(公告)号:US20230134779A1

    公开(公告)日:2023-05-04

    申请号:US18050058

    申请日:2022-10-27

    Applicant: Google LLC

    Abstract: A method includes obtaining a first frame of image data including a plurality of pixels. Each pixel of the plurality of pixels is associated with a respective color value and a respective depth value. The first frame renders a scene from a first point of view. The method includes generating a three-dimensional (3D) polygon mesh using the plurality of pixels and the respective depth values. The 3D polygon mesh includes a plurality of portions. Each respective portion defines a respective plurality of vertices defining a respective mesh density representative of a density of the respective plurality of vertices. Each vertex is associated with a corresponding pixel of the first frame of image data. The method includes generating a second frame of image data via reprojection using the generated 3D polygon mesh. The second frame of image data has a second point of view different from the first point of view.

    Spatially adaptive video compression for multiple streams of color and depth

    公开(公告)号:US10757410B1

    公开(公告)日:2020-08-25

    申请号:US16523247

    申请日:2019-07-26

    Applicant: GOOGLE LLC

    Abstract: Techniques of compressing color video images include computing a delta quantization parameter (ΔQP) for the color images based on a similarity between the depth image surface normal and the view direction associated with a color image. For example, upon receiving a frame having an image with multiple color and depth images, a computer finds a depth image that is closest in orientation to a color image. For each pixel of that depth image, the computer generates a blend weight based on an orientation of a normal to a position of the depth image and the viewpoints from which the plurality of color images were captured. The computer then generates a value of ΔQP based on the blend weight and determines a macroblock of color image corresponding to the position, the macroblock being associated with the value of ΔQP for the pixel.

    Image-based geometric fusion of multiple depth images using ray casting

    公开(公告)号:US10818071B1

    公开(公告)日:2020-10-27

    申请号:US16523702

    申请日:2019-07-26

    Applicant: Google LLC

    Abstract: Techniques of rendering images includes generating signed distance values (SDVs) along a ray from a specified viewpoint in terms of projected distances along that ray from given depth images. For each pixel in an image of from the perspective of the specified viewpoint, a ray is traced into the three-dimensional scene represented by the image. An iterative step is performed along the ray, obtaining in each iteration a three-dimensional world-space point p. The result is the signed distance sj as measured from depth view Dj. If the absolute value of the signed distance sj is greater than some truncation threshold parameter, the signed distance sj is replaced by a special undefined value. The defined signed-distance values are aggregated to obtain an overall signed distance s. Finally, the roots or zero set (isosurface) of the signed distance field is determined.

    Neural rerendering from 3D models

    公开(公告)号:US11288857B2

    公开(公告)日:2022-03-29

    申请号:US16837612

    申请日:2020-04-01

    Applicant: Google LLC

    Abstract: According to an aspect, a method for neural rerendering includes obtaining a three-dimensional (3D) model representing a scene of a physical space, where the 3D model is constructed from a collection of input images, rendering an image data buffer from the 3D model according to a viewpoint, where the image data buffer represents a reconstructed image from the 3D model, receiving, by a neural rerendering network, the image data buffer, receiving, by the neural rerendering network, an appearance code representing an appearance condition, and transforming, by the neural rerendering network, the image data buffer into a rerendered image with the viewpoint of the image data buffer and the appearance condition specified by the appearance code.

    NEURAL RERENDERING FROM 3D MODELS
    5.
    发明申请

    公开(公告)号:US20200320777A1

    公开(公告)日:2020-10-08

    申请号:US16837612

    申请日:2020-04-01

    Applicant: Google LLC

    Abstract: According to an aspect, a method for neural rerendering includes obtaining a three-dimensional (3D) model representing a scene of a physical space, where the 3D model is constructed from a collection of input images, rendering an image data buffer from the 3D model according to a viewpoint, where the image data buffer represents a reconstructed image from the 3D model, receiving, by a neural rerendering network, the image data buffer, receiving, by the neural rerendering network, an appearance code representing an appearance condition, and transforming, by the neural rerendering network, the image data buffer into a rerendered image with the viewpoint of the image data buffer and the appearance condition specified by the appearance code.

Patent Agency Ranking