公开(公告)号：US20240161484A1

公开(公告)日：2024-05-16

申请号：US18418386

申请日：2024-01-22

Applicant: SHANGHAITECH UNIVERSITY

Inventor： Peihao WANG ,  Jiakai ZHANG ,  Xinhang LIU ,  Zhijie LIU ,  Jingyi YU

IPC: G06V10/82 ,  G06T7/37

CPC classification number: G06V10/82 ,  G06T7/37 ,  G06T2207/20084

Abstract: A computer-implemented method is provided. The method includes obtaining a plurality of images representing projections of an object placed in a plurality of poses and a plurality of translations; assigning a pose embedding vector, a flow embedding vector and a contrast transfer function (CTF) embedding vector to each image; encoding, by a computer device, a machine learning model comprising a pose network, a flow network, a density network and a CTF network; training the machine learning model using the plurality of images; and reconstructing a 3D structure of the object based on the trained machine learning module.

公开(公告)号：US20240412377A1

公开(公告)日：2024-12-12

申请号：US18542803

申请日：2023-12-18

Applicant: SHANGHAITECH UNIVERSITY

Inventor： Peihao WANG ,  Jiakai ZHANG ,  Xinhang LIU ,  Zhijie LIU ,  Jingyi YU

IPC: G06T7/149 ,  G06T9/00 ,  G06T15/10

Abstract: Described herein are methods and non-transitory computer-readable media of a computing system configured to obtain a plurality of images of an object from a plurality of orientations at a plurality of times. A machine learning model is encoded to represent a continuous density field of the object that maps a spatial coordinate to a density value. The machine learning model comprises a deformation module configured to deform the spatial coordinate in accordance with a timestamp and a trained deformation weight. The machine learning model further comprises a neural radiance module configured to derive the density value in accordance with the deformed spatial coordinate, the timestamp, a direction, and a trained radiance weight. The machine learning model is trained using the plurality of images. A three-dimensional structure of the object is constructed based on the trained machine learning model.

公开(公告)号：US20240290059A1

公开(公告)日：2024-08-29

申请号：US18571748

申请日：2021-07-26

Applicant: SHANGHAITECH UNIVERSITY

Inventor： Jiakai ZHANG ,  Jingyi YU ,  Lan XU

IPC: G06V10/25 ,  G06T3/02 ,  G06T7/50

CPC classification number: G06V10/25 ,  G06T3/02 ,  G06T7/50

Abstract: A computer-implemented method of generating editable free-viewport videos is provided. A plurality of video of a scene from a plurality of views is obtained. The scene comprises includes an environment and one or more dynamic entities. A 3D bounding-box is generated for each dynamic entity in the scene. A computer device encodes a machine learning model including an environment layer and a dynamic entity layer for each dynamic entity in the scene. The environment layer represents a continuous function of space and time of the environment. The dynamic entity layer represents a continuous function of space and time of the dynamic entity. The dynamic entity layer includes a deformation module and a neural radiance module. The deformation module is configured to deform a spatial coordinate in accordance with a timestamp and a trained deformation weight. The neural radiance module is configured to derive a density value and a color.