-
公开(公告)号:US11880935B2
公开(公告)日:2024-01-23
申请号:US17951405
申请日:2022-09-23
Applicant: SHANGHAITECH UNIVERSITY
IPC: G06T17/00 , G06V10/82 , G06F18/214 , G06F18/213 , G06T15/04 , G06T15/20 , G06T17/20 , G06V20/64 , G06T7/593 , H04N13/207
CPC classification number: G06T17/00 , G06F18/213 , G06F18/214 , G06T7/593 , G06T15/04 , G06T15/205 , G06T17/20 , G06V10/82 , G06V20/653 , H04N13/207 , G06T2200/08 , G06T2207/10016 , G06T2207/10024 , G06T2207/10028 , G06T2207/20081 , G06T2207/20084 , G06T2207/30196 , G06T2210/56
Abstract: An image-based method of modeling and rendering a three-dimensional model of an object is provided. The method comprises: obtaining a three-dimensional point cloud at each frame of a synchronized, multi-view video of an object, wherein the video comprises a plurality of frames; extracting a feature descriptor for each point in the point cloud for the plurality of frames without storing the feature descriptor for each frame; producing a two-dimensional feature map for a target camera; and using an anti-aliased convolutional neural network to decode the feature map into an image and a foreground mask.
-
公开(公告)号:US12270910B2
公开(公告)日:2025-04-08
申请号:US17884273
申请日:2022-08-09
Applicant: XIAMEN UNIVERSITY , SHANGHAITECH UNIVERSITY
Inventor: Cheng Wang , Jialian Li , Lan Xu , Chenglu Wen , Jingyi Yu
Abstract: Described herein are systems and methods for training machine learning models to generate three-dimensional (3D) motions based on light detection and ranging (LiDAR) point clouds. In various embodiments, a computing system can encode a machine learning model representing an object in a scene. The computing system can train the machine learning model using a dataset comprising synchronous LiDAR point clouds captured by monocular LiDAR sensors and ground-truth three-dimensional motions obtained from IMU devices. The machine learning model can be configured to generate a three-dimensional motion of the object based on an input of a plurality of point cloud frames captured by a monocular LiDAR sensor.
-
公开(公告)号:US11410459B2
公开(公告)日:2022-08-09
申请号:US16894032
申请日:2020-06-05
Applicant: SHANGHAITECH UNIVERSITY
Inventor: Zhiru Shi , Minye Wu , Wenguang Ma , Jingyi Yu
IPC: H04N5/225 , G06V20/64 , H04N13/00 , G06V40/16 , H04N13/282
Abstract: A method of detecting and recognizing faces using a light field camera array is provided. The method includes capturing multi-view color images using the light field camera array; obtaining a depth map; conducting light field rendering using a weight function comprising a depth component and a sematic component, where the weight function assigns a ray in the light field with a weight; and detecting and recognizing a face.
-
公开(公告)号:US20230071559A1
公开(公告)日:2023-03-09
申请号:US17980754
申请日:2022-11-04
Applicant: SHANGHAITECH UNIVERSITY
Abstract: A method of rendering an object is provided. The method comprises: encoding a feature vector to each point in a point cloud for an object, wherein the feature vector comprises an alpha matte; projecting each point in the point cloud and the corresponding feature vector to a target view to compute a feature map; and using a neural rendering network to decode the feature map into a RGB image and the alpha matte and to update the feature vector.
-
公开(公告)号:US11861840B2
公开(公告)日:2024-01-02
申请号:US17219555
申请日:2021-03-31
Applicant: SHANGHAITECH UNIVERSITY
Inventor: Ziran Xing , Zhiru Shi , Yi Ma , Jingyi Yu
CPC classification number: G06T7/11 , G06T7/162 , G06T7/187 , G06T7/50 , G06T2207/10028
Abstract: According to some embodiments, an imaging processing method for extracting a plurality of planar surfaces from a depth map includes computing a depth change indication map (DCI) from a depth map in accordance with a smoothness threshold. The imaging processing method further includes recursively extracting a plurality of planar region from the depth map, wherein the size of each planar region is dynamically adjusted according to the DCI. The imaging processing method further includes clustering the extracted planar regions into a plurality of groups in accordance with a distance function; and growing each group to generate pixel-wise segmentation results and inlier points statistics simultaneously.
-
公开(公告)号:US10762654B2
公开(公告)日:2020-09-01
申请号:US16675617
申请日:2019-11-06
Applicant: SHANGHAITECH UNIVERSITY
Inventor: Jingyi Yu
Abstract: A method of generating a three-dimensional model of an object is disclosed. The method may use a light field camera to capture a plurality of light field images at a plurality of viewpoints. The method may include capturing a first light field image at a first viewpoint; capturing a second light field image at the second viewpoint; estimating a rotation and a translation of a light field from the first viewpoint to the second viewpoint; obtaining a disparity map from each of the plurality of light field image; and computing a three-dimensional point cloud by optimizing the rotation and translation of the light field and the disparity map. The first light field image may include a first plurality of subaperture images and the second light field image may include a second plurality of subaperture images.
-
公开(公告)号:US11880964B2
公开(公告)日:2024-01-23
申请号:US17074123
申请日:2020-10-19
Applicant: SHANGHAITECH UNIVERSITY
CPC classification number: G06T5/50 , G06T7/557 , G06T7/596 , G06T2200/21 , G06T2207/10052
Abstract: A method of processing light field images for separating a transmitted layer from a reflection layer. The method comprises capturing a plurality of views at a plurality of viewpoints with different polarization angles; obtaining an initial disparity estimation for a first view using SIFT-flow, and warping the first view to a reference view; optimizing an objective function comprising a transmitted layer and a secondary layer using an Augmented Lagrange Multiplier (ALM) with Alternating Direction Minimizing (ADM) strategy; updating the disparity estimation for the first view; repeating the steps of optimizing the objective function and updating the disparity estimation until the change in the objective function between two consecutive iterations is below a threshold; and separating the transmitted layer and the secondary layer using the disparity estimation for the first view.
-
公开(公告)号:US20230273315A1
公开(公告)日:2023-08-31
申请号:US17884406
申请日:2022-08-09
Applicant: XIAMEN UNIVERSITY , SHANGHAITECH UNIVERSITY
Inventor: Chenglu WEN , Yudi Dai , Lan Xu , Cheng Wang , Jingyi Yu
CPC classification number: G01S17/58 , G01S17/89 , G06T19/006
Abstract: Described herein are systems and methods of capturing motions of humans in a scene. A plurality of IMU devices and a LiDAR sensor are mounted on a human. IMU data is captured by the IMU devices and LiDAR data is captured by the LiDAR sensor. Motions of the human are estimated based on the IMU data and the LiDAR data. A three-dimensional scene map is built based on the LiDAR data. An optimization is performed to obtain optimized motions of the human and optimized scene map.
-
公开(公告)号:US11727628B2
公开(公告)日:2023-08-15
申请号:US17980754
申请日:2022-11-04
Applicant: SHANGHAITECH UNIVERSITY
CPC classification number: G06T15/205 , G06T7/194 , G06T7/55 , G06T7/80 , G06T15/20 , G06T15/503 , G06T2207/10028 , G06T2207/20081 , G06T2207/20084 , G06T2210/52 , G06T2210/56
Abstract: A method of rendering an object is provided. The method comprises: encoding a feature vector to each point in a point cloud for an object, wherein the feature vector comprises an alpha matte; projecting each point in the point cloud and the corresponding feature vector to a target view to compute a feature map; and using a neural rendering network to decode the feature map into a RGB image and the alpha matte and to update the feature vector.
-
-
-
-
-
-
-
-