公开(公告)号：US20210042898A1

公开(公告)日：2021-02-11

申请号：US17072397

申请日：2020-10-16

Applicant: SHANGHAITECH UNIVERSITY

Inventor： Minye WU ,  Zhiru SHI

IPC: G06T5/50 ,  G06T7/55 ,  G06T3/00 ,  G06T5/00 ,  G06T7/80

Abstract: A method of image processing for occlusion removal in images and videos captured by light field camera systems. The method comprises: capturing a plurality of camera views using a plurality of cameras; capturing a plurality of depth maps using a plurality of depth sensors; generating a depth map for each camera view; calculating a target view on a focal plane corresponding to a virtual camera; set a weighting function on the pixels on the camera views based on the depth map and a virtual distance; and blending the pixels in accordance with the weighting function to generate a refocused target view.

公开(公告)号：US20200279387A1

公开(公告)日：2020-09-03

申请号：US16877471

申请日：2020-05-18

Applicant: SHANGHAITECH UNIVERSITY

Inventor： Yingliang ZHANG ,  Guli ZHANG ,  Zhiru SHI

IPC: G06T7/557 ,  G06K9/62 ,  G06N3/08 ,  G06F17/18

Abstract: A light field image processing method is disclosed for removing occluding foreground and blurring uninterested objects, by differentiating objects located at different depths of field and objects belonging to distinct categories, to create see-through effects. In various embodiments, the image processing method may blur a background object behind a specified object of interest. The image processing method may also at least partially remove from the rendered image any occluding object that may prevent a viewer from viewing the object of interest. The image processing method may further blur areas of the rendered image that represent an object in the light field other than the object of interest. The method includes steps of constructing a light field weight function comprising a depth component and a semantic component, where the weight function assigns a ray in the light field with a weight; and conducting light field rendering using the weight function.

公开(公告)号：US20200296419A1

公开(公告)日：2020-09-17

申请号：US16891622

申请日：2020-06-03

Applicant: SHANGHAITECH UNIVERSITY

Inventor： Zhiru SHI ,  Qiang HU

IPC: H04N19/597 ,  H04N19/167 ,  H04N19/172 ,  G06K9/00

Abstract: A method of perceptual video coding based on face detection is provided. The method includes calculating a bit allocation scheme for coding a light field video based on a saliency map of the face, calculating an LCU level Lagrange multiplier for coding a light field video based on a saliency map of the face and calculating an LCU level quantization parameter for coding a light field video based on a saliency map of the face.

公开(公告)号：US20210241462A1

公开(公告)日：2021-08-05

申请号：US17219555

申请日：2021-03-31

Applicant: SHANGHAITECH UNIVERSITY

Inventor： Ziran XING ,  Zhiru SHI ,  Yi MA ,  Jingyi YU

IPC: G06T7/11 ,  G06T7/50 ,  G06T7/187 ,  G06T7/162

Abstract: According to some embodiments, an imaging processing method for extracting a plurality of planar surfaces from a depth map includes computing a depth change indication map (DCI) from a depth map in accordance with a smoothness threshold. The imaging processing method further includes recursively extracting a plurality of planar region from the depth map, wherein the size of each planar region is dynamically adjusted according to the DCI. The imaging processing method further includes clustering the extracted planar regions into a plurality of groups in accordance with a distance function; and growing each group to generate pixel-wise segmentation results and inlier points statistics simultaneously.

公开(公告)号：US20210082096A1

公开(公告)日：2021-03-18

申请号：US17074123

申请日：2020-10-19

Applicant: SHANGHAITECH UNIVERSITY

Inventor： Minye WU ,  Zhiru SHI ,  Jingyi YU

IPC: G06T5/50 ,  G06T7/557 ,  G06T7/593

Abstract: A method of processing light field images for separating a transmitted layer from a reflection layer. The method comprises capturing a plurality of views at a plurality of viewpoints with different polarization angles; obtaining an initial disparity estimation for a first view using SIFT-flow, and warping the first view to a reference view; optimizing an objective function comprising a transmitted layer and a secondary layer using an Augmented Lagrange Multiplier (ALM) with Alternating Direction Minimizing (ADM) strategy; updating the disparity estimation for the first view; repeating the steps of optimizing the objective function and updating the disparity estimation until the change in the objective function between two consecutive iterations is below a threshold; and separating the transmitted layer and the secondary layer using the disparity estimation for the first view.

公开(公告)号：US20200302155A1

公开(公告)日：2020-09-24

申请号：US16894032

申请日：2020-06-05

Applicant: SHANGHAITECH UNIVERSITY

Inventor： Zhiru SHI ,  Minye WU ,  Wengguang MA ,  Jingyi YU

IPC: G06K9/00 ,  H04N13/282 ,  H04N5/225

Abstract: A method of detecting and recognizing faces using a light field camera array is provided. The method includes capturing multi-view color images using the light field camera array; obtaining a depth map; conducting light field rendering using a weight function comprising a depth component and a sematic component, where the weight function assigns a ray in the light field with a weight; and detecting and recognizing a face.