公开(公告)号：US11195259B2

公开(公告)日：2021-12-07

申请号：US16703712

申请日：2019-12-04

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingmao Li ,  Chenchi Luo ,  George Q. Chen ,  Youngjun Yoo

IPC: G06T5/00 ,  G06T5/50 ,  G06T7/55 ,  G06T7/73 ,  G06T3/00 ,  G06T5/40 ,  G06T7/593

Abstract: A method includes obtaining, using first and second image sensors of an electronic device, first and second images, respectively, of a scene. The method also includes obtaining, using an image depth sensor of the electronic device, a third image and a first depth map of the scene, the first depth map having a resolution lower than a resolution of the first and second images. The method further includes undistorting the first and second images using the third image and the first depth map. The method also includes rectifying the first and second images using the third image and the first depth map. The method further includes generating a disparity map using the first and second images that have been undistorted and rectified. In addition, the method includes generating a second depth map using the disparity map and the first depth map, where the second depth map has a resolution that is higher than the resolution of the first depth map.

公开(公告)号：US20210281813A1

公开(公告)日：2021-09-09

申请号：US16811585

申请日：2020-03-06

Applicant: Samsung Electronics Co., Ltd.

Inventor： Kushal K. Vyas ,  Yingmao Li ,  Chenchi Luo ,  George Q. Chen ,  Hamid R. Sheikh ,  Youngjun Yoo ,  Michael O. Polley

IPC: H04N13/128 ,  G06T3/40 ,  G06T5/00 ,  G06T5/50

Abstract: A method includes obtaining, using at least one processor, first and second input image frames, where the first and second input image frames are associated with first and second image planes, respectively. The method also includes obtaining, using the at least one processor, a depth map associated with the first input image frame. The method further includes producing another version of the depth map by performing one or more times: (a) projecting, using the at least one processor, the first input image frame to the second image plane in order to produce a projected image frame using (i) the depth map and (ii) information identifying a conversion from the first image plane to the second image plane and (b) adjusting, using the at least one processor, at least one of the depth map and the information identifying the conversion from the first image plane to the second image plane.

公开(公告)号：US20200267331A1

公开(公告)日：2020-08-20

申请号：US16369071

申请日：2019-03-29

Applicant: Samsung Electronics, Co. Ltd.

Inventor： David Liu ,  Michael Polley ,  Youngjun F. Yoo ,  Hamid Rahim Sheikh ,  Kaimo Lin ,  George Chen ,  Chenchi Luo

IPC: H04N5/265 ,  G06F3/01 ,  G06T7/00 ,  H04N5/232

Abstract: A method of capturing a photo using a mobile device is described. The method comprises detecting a signature motion of the mobile device; capturing video using a camera of the mobile device upon detecting the signature motion; and automatically identifying a start frame and an end frame of the captured images using neural networks. A mobile device for capturing a photo is also described.

公开(公告)号：US11823353B2

公开(公告)日：2023-11-21

申请号：US17106676

申请日：2020-11-30

Applicant: Samsung Electronics Co., Ltd.

Inventor： Chenchi Luo ,  Yingmao Li ,  Kaimo Lin ,  Youngjun Yoo

IPC: G06T5/00 ,  G06T7/593 ,  G06T5/50 ,  G06N3/08

CPC classification number: G06T5/002 ,  G06N3/08 ,  G06T5/50 ,  G06T7/593 ,  G06T2207/10012 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: A method includes processing, using at least one processor of an electronic device, multiple reference images of a scene using a first convolutional neural network (CNN) to generate a confidence map and a disparity map. The method also includes generating, using the at least one processor, an initial Bokeh image based on the disparity map and the reference images using a depth-of-field (DoF) renderer. The method further includes refining, using the at least one processor, the initial Bokeh image using a second CNN to generate a refined Bokeh image, where the second CNN uses the confidence map, the disparity map, and the reference images to generate the refined Bokeh image.

公开(公告)号：US20220036513A1

公开(公告)日：2022-02-03

申请号：US17106676

申请日：2020-11-30

Applicant: Samsung Electronics Co., Ltd.

Inventor： Chenchi Luo ,  Yingmao Li ,  Kaimo Lin ,  Youngjun Yoo

IPC: G06T5/00 ,  G06T7/593 ,  G06T5/50 ,  G06N3/08

Abstract: A method includes processing, using at least one processor of an electronic device, multiple reference images of a scene using a first convolutional neural network (CNN) to generate a confidence map and a disparity map. The method also includes generating, using the at least one processor, an initial Bokeh image based on the disparity map and the reference images using a depth-of-field (DoF) renderer. The method further includes refining, using the at least one processor, the initial Bokeh image using a second CNN to generate a refined Bokeh image, where the second CNN uses the confidence map, the disparity map, and the reference images to generate the refined Bokeh image.

公开(公告)号：US20210390375A1

公开(公告)日：2021-12-16

申请号：US17135573

申请日：2020-12-28

Applicant: Samsung Electronics Co., Ltd.

Inventor： Chenchi Luo ,  Gyeongmin Choe ,  Yingmao Li ,  Zeeshan Nadir ,  Hamid R. Sheikh ,  John Seokjun Lee ,  Youngjun Yoo

IPC: G06N3/04 ,  G06T7/60 ,  G06T7/44 ,  G06N3/08

Abstract: A method includes obtaining, using at least one processor of an electronic device, multiple calibration parameters associated with multiple sensors of a selected mobile device. The method also includes obtaining, using the at least one processor, an identification of multiple imaging tasks. The method further includes obtaining, using the at least one processor, multiple synthetically-generated scene images. In addition, the method includes generating, using the at least one processor, multiple training images and corresponding meta information based on the calibration parameters, the identification of the imaging tasks, and the scene images. The training images and corresponding meta information are generated concurrently, different ones of the training images correspond to different ones of the sensors, and different pieces of the meta information correspond to different ones of the imaging tasks.

公开(公告)号：US20210342998A1

公开(公告)日：2021-11-04

申请号：US17246163

申请日：2021-04-30

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingmao Li ,  Chenchi Luo ,  Youngjun Yoo

IPC: G06T7/00 ,  G06T7/215 ,  G06T7/246 ,  G06T7/579 ,  G06T11/00 ,  G06T3/00 ,  G06T7/254

Abstract: A method includes obtaining at least one image and a ground truth map associated with the at least one image. The method also includes generating multiple optical maps using multiple algorithms and the at least one image. The method further includes, for each algorithm, identifying at least one score for the algorithm using one or more of the optical maps generated using the algorithm and the ground truth map. The ground truth map identifies one or more boundaries associated with one or more foreground objects in the at least one image. The scores identify how well the optical maps generated using the algorithms separate the one or more foreground objects from a background in the at least one image.

公开(公告)号：US11132772B2

公开(公告)日：2021-09-28

申请号：US16712882

申请日：2019-12-12

Applicant: Samsung Electronics Co., Ltd.

Inventor： Chenchi Luo ,  Yingmao Li ,  Youngjun Yoo ,  George Q. Chen ,  Kaimo Lin ,  David D. Liu ,  Gyeongmin Choe

IPC: G06T5/00 ,  G06T7/80 ,  H04N13/128

Abstract: A method includes obtaining a first image of a scene using a first image sensor of an electronic device and a second image of the scene using a second image sensor of the electronic device. The method also includes generating a first feature map from the first image and a second feature map from the second image. The method further includes generating a third feature map based on the first feature map, the second feature map, and an asymmetric search window. The method additionally includes generating a depth map by restoring spatial resolution to the third feature map.

公开(公告)号：US11720782B2

公开(公告)日：2023-08-08

申请号：US17135573

申请日：2020-12-28

Applicant: Samsung Electronics Co., Ltd.

Inventor： Chenchi Luo ,  Gyeongmin Choe ,  Yingmao Li ,  Zeeshan Nadir ,  Hamid R. Sheikh ,  John Seokjun Lee ,  Youngjun Yoo

IPC: G06N3/00 ,  G06N3/045 ,  G06T7/60 ,  G06T7/44 ,  G06N3/08 ,  G06N3/042

CPC classification number: G06N3/045 ,  G06N3/042 ,  G06N3/08 ,  G06T7/44 ,  G06T7/60 ,  G06T2207/10024 ,  G06T2207/20081

Abstract: A method includes obtaining, using at least one processor of an electronic device, multiple calibration parameters associated with multiple sensors of a selected mobile device. The method also includes obtaining, using the at least one processor, an identification of multiple imaging tasks. The method further includes obtaining, using the at least one processor, multiple synthetically-generated scene images. In addition, the method includes generating, using the at least one processor, multiple training images and corresponding meta information based on the calibration parameters, the identification of the imaging tasks, and the scene images. The training images and corresponding meta information are generated concurrently, different ones of the training images correspond to different ones of the sensors, and different pieces of the meta information correspond to different ones of the imaging tasks.

公开(公告)号：US11580399B2

公开(公告)日：2023-02-14

申请号：US16400007

申请日：2019-04-30

Applicant: Samsung Electronics Co., Ltd.

Inventor： Chenchi Luo ,  Manish Goel ,  David Liu

IPC: G06N3/08 ,  G06N3/082 ,  G06F17/15 ,  G06N3/04

Abstract: An electronic device, method, and computer readable medium for 3D association of detected objects are provided. The electronic device includes a memory and at least one processor coupled to the memory. The at least one processor configured to convolve an input to a neural network with a basis kernel to generate a convolution result, scale the convolution result by a scalar to create a scaled convolution result, and combine the scaled convolution result with one or more of a plurality of scaled convolution results to generate an output feature map.