公开(公告)号：US11250571B2

公开(公告)日：2022-02-15

申请号：US16566155

申请日：2019-09-10

Applicant: Apple Inc.

Inventor： Alexander Lindskog ,  Michael W. Tao ,  Alexandre Naaman

IPC: G06T7/11 ,  G06T5/00 ,  G06K9/72 ,  G06K9/00

Abstract: This disclosure relates to techniques for the robust usage of semantic segmentation information in image processing techniques, e.g., shallow depth of field (SDOF) renderings. Semantic segmentation may be defined as a process of creating a mask over an image, wherein pixels are segmented into a predefined set of semantic classes. Segmentations may be binary (e.g., a ‘person pixel’ or a ‘non-person pixel’) or multi-class (e.g., a pixel may be labelled as: ‘person,’ ‘dog,’ ‘cat,’ etc.). As semantic segmentation techniques grow in accuracy and adoption, it is becoming increasingly important to develop methods of utilizing such segmentations and developing flexible techniques for integrating segmentation information into existing computer vision applications, such as synthetic SDOF renderings, to yield improved results in a wide range of image capture scenarios. In some embodiments, a refinement operation may be employed on a camera device's initial depth, disparity and/or blur estimates that leverages semantic segmentation information.

公开(公告)号：US10762655B1

公开(公告)日：2020-09-01

申请号：US16567581

申请日：2019-09-11

Applicant: Apple Inc.

Inventor： Alexander Lindskog ,  Michael W. Tao ,  Mark N. Jouppi

IPC: H04N13/271 ,  G06T7/571 ,  H04N5/369

Abstract: The disclosure pertains to techniques for image processing. One such technique comprises a method for image processing comprising obtaining first light information from a set of light-sensitive pixels for a scene, the pixels including phase detection (PD) pixels and non-PD pixels, generating a first PD pixel image from the first light information, the first PD pixel image having a first resolution, generating a higher resolution image from the plurality of non-PD pixels, wherein the higher resolution image has a resolution greater than the resolution of the first PD pixel image, matching a first pixel of the first PD pixel image to the higher resolution image, wherein the matching is based on a set of correlations between the first pixel and non-PD pixel within a predetermined distance of the first pixel, and determining a disparity map for an image associated with the first light information, based on the match.

公开(公告)号：US20170070720A1

公开(公告)日：2017-03-09

申请号：US14864650

申请日：2015-09-24

Applicant: Apple Inc.

Inventor： Thomas E. Bishop ,  Alexander Lindskog ,  Claus Molgaard ,  Frank Doepke

IPC: H04N13/00 ,  G06T5/00 ,  G06T7/00 ,  H04N13/02

CPC classification number: H04N13/106 ,  G06T5/002 ,  G06T7/593 ,  H04N5/23229 ,  H04N5/2356

Abstract: Generating an image with a selected level of background blur includes capturing, by a first image capture device, a plurality of frames of a scene, wherein each of the plurality of frames has a different focus depth, obtaining a depth map of the scene, determining a target object and a background in the scene based on the depth map, determining a goal blur for the background, and selecting, for each pixel in an output image, a corresponding pixel from the focus stack.

Abstract translation: 生成具有所选择的背景模糊级别的图像包括由第一图像捕获装置捕获场景的多个帧，其中多个帧中的每一个具有不同的焦点深度，获得场景的深度图，确定 基于深度图的场景中的目标对象和背景，确定背景的目标模糊，以及从输出图像中的每个像素选择来自焦点堆栈的对应像素。

公开(公告)号：US20160360091A1

公开(公告)日：2016-12-08

申请号：US14864565

申请日：2015-09-24

Applicant: Apple Inc.

Inventor： Alexander Lindskog ,  Frank Doepke ,  Ralf Brunner ,  Thomas E. Bishop

IPC: H04N5/232 ,  H04N5/225 ,  G06T7/00

CPC classification number: H04N5/23212 ,  G02B7/36 ,  G02B27/0075 ,  G06T7/50 ,  H04N5/2254 ,  H04N5/23293 ,  H04N5/2356 ,  H04N5/357

Abstract: Generating a focus stack, including receiving initial focus data that identifies a plurality of target depths, positioning a lens at a first position to capture a first image at a first target depth of the plurality of target depths, determining, in response to capturing the first image and prior to capturing additional images, a sharpness metric for the first image, capturing, in response to determining that the sharpness metric for the first image is an unacceptable value, a second image at a second position based on the sharpness metric, wherein the second position is not included in the plurality of target depths, determining that a sharpness metric for the second image is an acceptable value, and generating a focus stack using the second image.

Abstract translation: 生成焦点堆叠，包括接收识别多个目标深度的初始聚焦数据，将透镜定位在第一位置以在多个目标深度的第一目标深度捕获第一图像，响应于捕获第一 图像，并且在捕获附加图像之前，针对第一图像的锐度度量，响应于确定第一图像的锐度度量是不可接受的值，捕获，基于锐度度量在第二位置处的第二图像，其中 第二位置不包括在多个目标深度中，确定第二图像的锐度度量是可接受的值，并且使用第二图像生成焦点堆叠。