-
公开(公告)号:US20240013479A1
公开(公告)日:2024-01-11
申请号:US18369904
申请日:2023-09-19
Applicant: SHANGHAITECH UNIVERSITY
Inventor: Minye WU , Chaolin RAO , Xin LOU , Pingqiang ZHOU , Jingyi YU
CPC classification number: G06T15/55 , G06T15/20 , G06T17/20 , G06T2210/56
Abstract: A computer-implemented method includes encoding a radiance field of an object onto a machine learning model; conducting, based on a set of training images of the object, a training process on the machine learning model to obtain a trained machine learning model, wherein the training process includes a first training process using a plurality of first test sample points followed by a second training process using a plurality of second test sample points located within a threshold distance from a surface region of the object; obtaining target view parameters indicating a view direction of the object; obtaining a plurality of rays associated with a target image of the object; obtaining render sample points on the plurality of rays associated with the target image; and rendering, by inputting the render sample points to the trained machine learning model, colors associated with the pixels of the target image.
-
公开(公告)号:US20190028693A1
公开(公告)日:2019-01-24
申请号:US16069181
申请日:2016-01-12
Applicant: SHANGHAITECH UNIVERSITY
IPC: H04N13/246 , H04N13/282 , H04N13/239 , H04N13/243 , G06T7/73 , G06T7/80 , G06T7/33
Abstract: A method of calibrating a camera array comprising a plurality of cameras configured to capture a plurality of images to generate a panorama, wherein the relative positions among the plurality of cameras are constant, the method comprising: moving the camera array from a first position to a second position; measuring a homogeneous transformation matrix of a reference point on the camera array between the first position and the second position; capturing images at the first position and the second position by a first camera and a second camera on the camera array; and determining a homogenous transformation matrix between the first camera and the second camera based on the images captured by the first camera and the second camera at the first position and the second position. The method further comprises identifying a feature in the images taken by the first camera at the first position and the second position, and estimating a rotation of the first camera from the first position to the second position based on the feature.
-
公开(公告)号:US20240296308A1
公开(公告)日:2024-09-05
申请号:US18646852
申请日:2024-04-26
Applicant: SHANGHAITECH UNIVERSITY
Inventor: Yueyang ZHENG , Chaolin RAO , Minye WU , Xin LOU , Pingqiang ZHOU , Jingyi YU
IPC: G06N3/04
CPC classification number: G06N3/04
Abstract: A computing system for encoding a machine learning model comprises a plurality of layers and a plurality of computation units. A first set of computation units are configured to process data at a first bit width. A second set of computation units are configured to process at a second bit width. The first bit width is higher than the second bit width. A memory is coupled to the computation units. A controller is coupled to the computation units and the memory. The controller is configured to provide instructions for encoding the machine learning model. The first set of computation units are configured to compute a first set of layers and the second set of computation units are configured to compute a second set of layers.
-
公开(公告)号:US20240161484A1
公开(公告)日:2024-05-16
申请号:US18418386
申请日:2024-01-22
Applicant: SHANGHAITECH UNIVERSITY
Inventor: Peihao WANG , Jiakai ZHANG , Xinhang LIU , Zhijie LIU , Jingyi YU
CPC classification number: G06V10/82 , G06T7/37 , G06T2207/20084
Abstract: A computer-implemented method is provided. The method includes obtaining a plurality of images representing projections of an object placed in a plurality of poses and a plurality of translations; assigning a pose embedding vector, a flow embedding vector and a contrast transfer function (CTF) embedding vector to each image; encoding, by a computer device, a machine learning model comprising a pose network, a flow network, a density network and a CTF network; training the machine learning model using the plurality of images; and reconstructing a 3D structure of the object based on the trained machine learning module.
-
公开(公告)号:US20200074658A1
公开(公告)日:2020-03-05
申请号:US16675617
申请日:2019-11-06
Applicant: Shanghaitech University
Inventor: Jingyi YU
Abstract: A method of generating a three-dimensional model of an object is disclosed. The method may use a light field camera to capture a plurality of light field images at a plurality of viewpoints. The method may include capturing a first light field image at a first viewpoint; capturing a second light field image at the second viewpoint; estimating a rotation and a translation of a light field from the first viewpoint to the second viewpoint; obtaining a disparity map from each of the plurality of light field image; and computing a three-dimensional point cloud by optimizing the rotation and translation of the light field and the disparity map. The first light field image may include a first plurality of subaperture images and the second light field image may include a second plurality of subaperture images.
-
公开(公告)号:US20240412377A1
公开(公告)日:2024-12-12
申请号:US18542803
申请日:2023-12-18
Applicant: SHANGHAITECH UNIVERSITY
Inventor: Peihao WANG , Jiakai ZHANG , Xinhang LIU , Zhijie LIU , Jingyi YU
Abstract: Described herein are methods and non-transitory computer-readable media of a computing system configured to obtain a plurality of images of an object from a plurality of orientations at a plurality of times. A machine learning model is encoded to represent a continuous density field of the object that maps a spatial coordinate to a density value. The machine learning model comprises a deformation module configured to deform the spatial coordinate in accordance with a timestamp and a trained deformation weight. The machine learning model further comprises a neural radiance module configured to derive the density value in accordance with the deformed spatial coordinate, the timestamp, a direction, and a trained radiance weight. The machine learning model is trained using the plurality of images. A three-dimensional structure of the object is constructed based on the trained machine learning model.
-
公开(公告)号:US20240281256A1
公开(公告)日:2024-08-22
申请号:US18646818
申请日:2024-04-26
Applicant: SHANGHAITECH UNIVERSITY
Inventor: Yuhan GU , Chaolin RAO , Minye WU , Xin LOU , Pingqiang ZHOU , Jingyi YU
CPC classification number: G06F9/3885 , G06T1/20 , G06T15/005
Abstract: A computing core for rendering an image computing core comprises a position encoding logic and a plurality of pipeline logics connected in series in a pipeline. The position encoding logic is configured to transform coordinates and directions of sampling points corresponding to a portion of the image into high dimensional representations. The plurality of pipeline logics are configured to output, based on the high dimensional representation of the coordinates and the high dimensional representation of the directions, intensity and color values of pixels corresponding to the portion of the image in one pipeline cycle. The plurality of pipeline logics are configured to run in parallel.
-
公开(公告)号:US20240202948A1
公开(公告)日:2024-06-20
申请号:US18574049
申请日:2021-07-05
Applicant: SHANGHAITECH UNIVERSITY
Inventor: Siyuan SHEN , Zi WANG , Shiying LI , Jingyi YU
IPC: G06T7/521 , G01S7/4865 , G01S17/42 , G01S17/894 , G06T7/557 , G06T7/77
CPC classification number: G06T7/521 , G01S7/4866 , G01S17/42 , G01S17/894 , G06T7/557 , G06T7/77 , G06T2207/10028 , G06T2207/10052 , G06T2207/20081 , G06T2207/20084
Abstract: A novel neural modeling framework Neural Transient Field (NeTF) is provided for non-line-of-sight (NLOS) imaging. NeTF recovers the 5D transient function in both spatial location and direction, and the training data input is parametrized on the spherical wave-fronts. A Markov chain Monte Carlo (MCMC) algorithm is used to account for sparse and unbalanced sampling in NeTF.
-
公开(公告)号:US20240161388A1
公开(公告)日:2024-05-16
申请号:US18281966
申请日:2021-04-13
Applicant: SHANGHAITECH UNIVERSITY
Inventor: Haimin LUO , Minye WU , Lan XU , Jingyi YU
CPC classification number: G06T15/20 , G06T7/596 , G06T7/62 , G06T17/00 , G06T2200/04 , G06T2207/20081 , G06T2207/20084
Abstract: A deep neural network based hair rendering system is presented to model high frequency component of furry objects. Compared with existing approaches, the present method can generate photo-realistic rendering results. An acceleration method is applied in our framework, which can speed up training and rendering processes. In addition, a patch-based training scheme is introduced, which significantly increases the quality of outputs and preserves high frequency details.
-
公开(公告)号:US20230273318A1
公开(公告)日:2023-08-31
申请号:US17884273
申请日:2022-08-09
Applicant: XIAMEN UNIVERSITY , SHANGHAITECH UNIVERSITY
Inventor: Cheng WANG , Jialian LI , Lan XU , Chenglu WEN , Jingyi YU
Abstract: Described herein are systems and methods for training machine learning models to generate three-dimensional (3D) motions based on light detection and ranging (LiDAR) point clouds. In various embodiments, a computing system can encode a machine learning model representing an object in a scene. The computing system can train the machine learning model using a dataset comprising synchronous LiDAR point clouds captured by monocular LiDAR sensors and ground-truth three-dimensional motions obtained from IMU devices. The machine learning model can be configured to generate a three-dimensional motion of the object based on an input of a plurality of point cloud frames captured by a monocular LiDAR sensor.
-
-
-
-
-
-
-
-
-