公开(公告)号：US11720782B2

公开(公告)日：2023-08-08

申请号：US17135573

申请日：2020-12-28

Applicant: Samsung Electronics Co., Ltd.

Inventor： Chenchi Luo ,  Gyeongmin Choe ,  Yingmao Li ,  Zeeshan Nadir ,  Hamid R. Sheikh ,  John Seokjun Lee ,  Youngjun Yoo

IPC: G06N3/00 ,  G06N3/045 ,  G06T7/60 ,  G06T7/44 ,  G06N3/08 ,  G06N3/042

CPC classification number: G06N3/045 ,  G06N3/042 ,  G06N3/08 ,  G06T7/44 ,  G06T7/60 ,  G06T2207/10024 ,  G06T2207/20081

Abstract: A method includes obtaining, using at least one processor of an electronic device, multiple calibration parameters associated with multiple sensors of a selected mobile device. The method also includes obtaining, using the at least one processor, an identification of multiple imaging tasks. The method further includes obtaining, using the at least one processor, multiple synthetically-generated scene images. In addition, the method includes generating, using the at least one processor, multiple training images and corresponding meta information based on the calibration parameters, the identification of the imaging tasks, and the scene images. The training images and corresponding meta information are generated concurrently, different ones of the training images correspond to different ones of the sensors, and different pieces of the meta information correspond to different ones of the imaging tasks.

公开(公告)号：US11532091B2

公开(公告)日：2022-12-20

申请号：US16946956

申请日：2020-07-13

Applicant: Samsung Electronics Co., Ltd.

Inventor： Kushal Kardam Vyas ,  Yingmao Li ,  Chenchi Luo ,  George Q. Chen ,  Hamid R. Sheikh ,  Youngjun Yoo ,  Michael O. Polley

IPC: G06T7/00 ,  G06T7/50 ,  G06T3/40 ,  G06T7/70 ,  G06T5/00

Abstract: A method includes obtaining, using at least one processor, an input image frame. The method also includes identifying, using the at least one processor, one or more regions of the input image frame containing redundant information. In addition, the method includes performing, using the at least one processor, an image processing task using the input image frame. The image processing task is guided based on the one or more identified regions of the input image frame. The method may further include obtaining, using the at least one processor, a coarse depth map associated with the input image frame. Performing the image processing task may include refining the coarse depth map to produce a refined depth map, where the refining of the coarse depth map is guided based on the one or more identified regions of the input image frame.

公开(公告)号：US11159731B2

公开(公告)日：2021-10-26

申请号：US16279724

申请日：2019-02-19

Applicant: Samsung Electronics Co., Ltd.

Inventor： David Liu ,  Youngjun Yoo

IPC: H04N5/232

Abstract: An electronic device, a method, and computer readable medium for operating an electronic device are disclosed. The method includes receiving data about a state of the electronic device from one or more sensors of the electronic device. The method also includes determining whether to modify a user interface button displayed on a display of the electronic device based on the received state data and parameters of a neural network. The method further includes modifying display of the user interface button on the display of the electronic device based on determining to modify. The method additionally includes providing, to the neural network, feedback data indicating whether the user interface button was triggered within a predetermined time period after modifying the user interface button.

公开(公告)号：US20210248722A1

公开(公告)日：2021-08-12

申请号：US16946989

申请日：2020-07-14

Applicant: Samsung Electronics Co., Ltd.

Inventor： Chenchi Luo ,  Yingmao Li ,  Youngjun Yoo

IPC: G06T5/00 ,  G06T5/50 ,  G06N3/04 ,  G06T3/40

Abstract: A method includes processing, using at least one processor of an electronic device, each of multiple images using a photometric augmentation engine, where the photometric augmentation engine performs one or more photometric augmentation operations. The method also includes applying, using the at least one processor, multiple layers of a convolutional neural network to each of the images, where each layer generates a corresponding feature map. The method further includes processing, using the at least one processor, at least one of the feature maps using at least one feature augmentation engine between consecutive layers of the multiple layers, where the at least one feature augmentation engine performs one or more feature augmentation operations.

公开(公告)号：US20210174479A1

公开(公告)日：2021-06-10

申请号：US16703712

申请日：2019-12-04

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingmao Li ,  Chenchi Luo ,  George Q. Chen ,  Youngjun Yoo

IPC: G06T5/00 ,  G06T5/40 ,  G06T5/50 ,  G06T7/55 ,  G06T7/73 ,  G06T3/00

Abstract: A method includes obtaining, using first and second image sensors of an electronic device, first and second images, respectively, of a scene. The method also includes obtaining, using an image depth sensor of the electronic device, a third image and a first depth map of the scene, the first depth map having a resolution lower than a resolution of the first and second images. The method further includes undistorting the first and second images using the third image and the first depth map. The method also includes rectifying the first and second images using the third image and the first depth map. The method further includes generating a disparity map using the first and second images that have been undistorted and rectified. In addition, the method includes generating a second depth map using the disparity map and the first depth map, where the second depth map has a resolution that is higher than the resolution of the first depth map.

公开(公告)号：US20210158142A1

公开(公告)日：2021-05-27

申请号：US16693112

申请日：2019-11-22

Applicant: Samsung Electronics Co., Ltd.

Inventor： Chenchi Luo ,  Yingmao Li ,  Youngjun Yoo

IPC: G06N3/08 ,  G06N3/04

Abstract: A method includes identifying, by at least one processor, multiple features of input data using a common feature extractor. The method also includes processing, by the at least one processor, at least some identified features using each of multiple pre-processing branches. Each pre-processing branch includes a first set of neural network layers and generates initial outputs associated with a different one of multiple data processing tasks. The method further includes combining, by the at least one processor, at least two initial outputs from at least two pre-processing branches to produce combined initial outputs. In addition, the method includes processing, by the at least one processor, at least some initial outputs or at least some combined initial outputs using each of multiple post-processing branches. Each post-processing branch includes a second set of neural network layers and generates final outputs associated with a different one of the multiple data processing tasks.

公开(公告)号：US20200267324A1

公开(公告)日：2020-08-20

申请号：US16279724

申请日：2019-02-19

Applicant: Samsung Electronics Co., Ltd.

Inventor： David Liu ,  Youngjun Yoo

IPC: H04N5/232

Abstract: An electronic device, a method, and computer readable medium for operating an electronic device are disclosed. The method includes receiving data about a state of the electronic device from one or more sensors of the electronic device. The method also includes determining whether to modify a user interface button displayed on a display of the electronic device based on the received state data and parameters of a neural network. The method further includes modifying display of the user interface button on the display of the electronic device based on determining to modify. The method additionally includes providing, to the neural network, feedback data indicating whether the user interface button was triggered within a predetermined time period after modifying the user interface button.

公开(公告)号：US20220014683A1

公开(公告)日：2022-01-13

申请号：US17484822

申请日：2021-09-24

Applicant: Samsung Electronics Co., Ltd.

Inventor： David Liu ,  Youngjun Yoo

IPC: H04N5/232

Abstract: An electronic device includes a display, a camera, and a processing device. The processing device is configured to determine whether (i) a user's face or eyes are within the camera's field of view or (ii) a gaze of the user is directed towards the display. The processing device is also configured, in response to determining that (i) the user's face or eyes are not within the camera's field of view or (ii) the gaze of the user is not directed towards the display, to modify a user interface button presented on the display. The user interface button may represent a shutter button configured to cause the camera or another camera of the electronic device to capture one or more images.

公开(公告)号：US11195259B2

公开(公告)日：2021-12-07

申请号：US16703712

申请日：2019-12-04

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingmao Li ,  Chenchi Luo ,  George Q. Chen ,  Youngjun Yoo

IPC: G06T5/00 ,  G06T5/50 ,  G06T7/55 ,  G06T7/73 ,  G06T3/00 ,  G06T5/40 ,  G06T7/593

Abstract: A method includes obtaining, using first and second image sensors of an electronic device, first and second images, respectively, of a scene. The method also includes obtaining, using an image depth sensor of the electronic device, a third image and a first depth map of the scene, the first depth map having a resolution lower than a resolution of the first and second images. The method further includes undistorting the first and second images using the third image and the first depth map. The method also includes rectifying the first and second images using the third image and the first depth map. The method further includes generating a disparity map using the first and second images that have been undistorted and rectified. In addition, the method includes generating a second depth map using the disparity map and the first depth map, where the second depth map has a resolution that is higher than the resolution of the first depth map.

公开(公告)号：US20210281813A1

公开(公告)日：2021-09-09

申请号：US16811585

申请日：2020-03-06

Applicant: Samsung Electronics Co., Ltd.

Inventor： Kushal K. Vyas ,  Yingmao Li ,  Chenchi Luo ,  George Q. Chen ,  Hamid R. Sheikh ,  Youngjun Yoo ,  Michael O. Polley

IPC: H04N13/128 ,  G06T3/40 ,  G06T5/00 ,  G06T5/50

Abstract: A method includes obtaining, using at least one processor, first and second input image frames, where the first and second input image frames are associated with first and second image planes, respectively. The method also includes obtaining, using the at least one processor, a depth map associated with the first input image frame. The method further includes producing another version of the depth map by performing one or more times: (a) projecting, using the at least one processor, the first input image frame to the second image plane in order to produce a projected image frame using (i) the depth map and (ii) information identifying a conversion from the first image plane to the second image plane and (b) adjusting, using the at least one processor, at least one of the depth map and the information identifying the conversion from the first image plane to the second image plane.