公开(公告)号：US12073507B2

公开(公告)日：2024-08-27

申请号：US17861199

申请日：2022-07-09

Applicant: Adobe Inc.

Inventor： Zexiang Xu ,  Zhixin Shu ,  Sai Bi ,  Qiangeng Xu ,  Kalyan Sunkavalli ,  Julien Philip

IPC: G06T15/20 ,  G06T15/06 ,  G06T15/80

CPC classification number: G06T15/205 ,  G06T15/06 ,  G06T15/80 ,  G06T2207/10028

Abstract: A scene modeling system receives a plurality of input two-dimensional (2D) images corresponding to a plurality of views of an object and a request to display a three-dimensional (3D) scene that includes the object. The scene modeling system generates an output 2D image for a view of the 3D scene by applying a scene representation model to the input 2D images. The scene representation model includes a point cloud generation model configured to generate, based on the input 2D images, a neural point cloud representing the 3D scene. The scene representation model includes a neural point volume rendering model configured to determine, for each pixel of the output image and using the neural point cloud and a volume rendering process, a color value. The scene modeling system transmits, responsive to the request, the output 2D image. Each pixel of the output image includes the respective determined color value.

公开(公告)号：US20240193850A1

公开(公告)日：2024-06-13

申请号：US18065456

申请日：2022-12-13

Applicant: Adobe Inc.

Inventor： Zhengfei Kuang ,  Fujun Luan ,  Sai Bi ,  Zhixin Shu ,  Kalyan K. Sunkavalli

IPC: G06T15/08 ,  G06T15/06 ,  G06T15/50 ,  G06T15/80 ,  G06T19/20

CPC classification number: G06T15/08 ,  G06T15/06 ,  G06T15/503 ,  G06T15/80 ,  G06T19/20 ,  G06T2219/2012

Abstract: Embodiments of the present disclosure provide systems, methods, and computer storage media for generating editable synthesized views of scenes by inputting image rays into neural networks using neural basis decomposition. In embodiments, a set of input images of a scene depicting at least one object are collected and used to generate a plurality of rays of the scene. The rays each correspond to three-dimensional coordinates and viewing angles taken from the images. A volume density of the scene is determined by inputting the three-dimensional coordinates from the neural radiance fields into a first neural network to generate a 3D geometric representation of the object. An appearance decomposition is produced by inputting the three-dimensional coordinates and the viewing angles of the rays into a second neural network.

公开(公告)号：US20240127402A1

公开(公告)日：2024-04-18

申请号：US18238290

申请日：2023-08-25

Applicant: Adobe Inc.

Inventor： Mohammad Reza Karimi Dastjerdi ,  Yannick Hold-Geoffroy ,  Sai Bi ,  Jonathan Eisenmann ,  Jean-François Lalonde

IPC: G06T5/50 ,  G06T5/00 ,  G06T15/50

CPC classification number: G06T5/50 ,  G06T5/002 ,  G06T15/506 ,  G06T2200/24 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20092 ,  G06T2207/20208

Abstract: In some examples, a computing system accesses a field of view (FOV) image that has a field of view less than 360 degrees and has low dynamic range (LDR) values. The computing system estimates lighting parameters from a scene depicted in the FOV image and generates a lighting image based on the lighting parameters. The computing system further generates lighting features generated the lighting image and image features generated from the FOV image. These features are aggregated into aggregated features and a machine learning model is applied to the image features and the aggregated features to generate a panorama image having high dynamic range (HDR) values.

公开(公告)号：US11669986B2

公开(公告)日：2023-06-06

申请号：US17233122

申请日：2021-04-16

Applicant: ADOBE INC.

Inventor： Sai Bi ,  Zexiang Xu ,  Kalyan Krishna Sunkavalli ,  David Jay Kriegman ,  Ravi Ramamoorthi

IPC: G06T7/514 ,  G06T17/20 ,  H04N13/111 ,  H04N13/282 ,  H04N13/128

CPC classification number: G06T7/514 ,  G06T17/20 ,  H04N13/111 ,  H04N13/128 ,  H04N13/282 ,  G06T2207/10012 ,  G06T2207/10028

Abstract: Enhanced methods and systems for generating both a geometry model and an optical-reflectance model (an object reconstruction model) for a physical object, based on a sparse set of images of the object under a sparse set of viewpoints. The geometry model is a mesh model that includes a set of vertices representing the object's surface. The reflectance model is SVBRDF that is parameterized via multiple channels (e.g., diffuse albedo, surface-roughness, specular albedo, and surface-normals). For each vertex of the geometry model, the reflectance model includes a value for each of the multiple channels. The object reconstruction model is employed to render graphical representations of a virtualized object (a VO based on the physical object) within a computation-based (e.g., a virtual or immersive) environment. Via the reconstruction model, the VO may be rendered from arbitrary viewpoints and under arbitrary lighting conditions.