公开(公告)号：US09667842B2

公开(公告)日：2017-05-30

申请号：US14474103

申请日：2014-08-30

Applicant: Apple Inc.

Inventor： Farhan A. Baqai ,  Claus Molgaard ,  Fabio Riccardi ,  Xuemei Zhang

IPC: H04N5/217 ,  H04N9/04 ,  H04N9/77 ,  G06T5/00

CPC classification number: H04N5/217 ,  G06T5/002 ,  G06T5/008 ,  G06T2207/20012 ,  G06T2207/20016 ,  G06T2207/20048 ,  H04N9/045

Abstract: Image enhancement is achieved by separating image signals, e.g. YCbCr image signals, into a series of frequency bands and performing locally-adaptive noise reduction on bands below a given frequency but not on bands above that frequency. The bands are summed to develop the image enhanced signals. The YCbCr, multi-band locally-adaptive approach to denoising is able to operate independently—and in an optimized fashion—on both luma and chroma channels. Noise reduction is done based on models developed for both luma and chroma channels by measurements taken for multiple frequency bands, in multiple patches on the ColorChecker chart, and at multiple gain levels, in order to develop a simple yet robust set of models that may be tuned off-line a single time for each camera and then applied to images taken by such cameras in real-time without excessive processing requirements and with satisfactory results across illuminant types and lighting conditions.

公开(公告)号：US09641820B2

公开(公告)日：2017-05-02

申请号：US14872098

申请日：2015-09-30

Applicant: Apple Inc.

Inventor： Farhan A. Baqai ,  Fabio Riccardi ,  Russell A. Pflughaupt ,  Claus Molgaard ,  Gijesh Varghese

IPC: H04N9/77 ,  H04N9/64 ,  H04N9/67 ,  H04N9/73 ,  H04N9/69

CPC classification number: H04N9/77 ,  G06K9/40 ,  G06T5/002 ,  G06T5/009 ,  G06T5/20 ,  G06T5/50 ,  G06T2207/10024 ,  G06T2207/20016 ,  G06T2207/20182 ,  G06T2207/20208 ,  G06T2207/20221 ,  H04N5/208 ,  H04N5/217 ,  H04N9/646 ,  H04N9/67 ,  H04N9/69 ,  H04N9/73

Abstract: Techniques for de-noising a digital image using a multi-band noise filter and a unique combination of texture and chroma metrics are described. A novel texture metric may be used during multi-band filter operations on an image's luma channel to determine if a given pixel is associated with a textured/smooth region of the image. A novel chroma metric may be used during the same multi-band filter operation to determine if the same pixel is associated with a blue/not-blue region of the image. Pixels identified as being associated with a smooth blue region may be aggressively de-noised and conservatively sharpened. Pixels identified as being associated with a textured blue region may be conservatively de-noised and aggressively sharpened. By coupling texture and chroma constraints it has been shown possible to mitigate noise in an image's smooth blue regions without affecting the edges/texture in other blue objects.

公开(公告)号：US20170070720A1

公开(公告)日：2017-03-09

申请号：US14864650

申请日：2015-09-24

Applicant: Apple Inc.

Inventor： Thomas E. Bishop ,  Alexander Lindskog ,  Claus Molgaard ,  Frank Doepke

IPC: H04N13/00 ,  G06T5/00 ,  G06T7/00 ,  H04N13/02

CPC classification number: H04N13/106 ,  G06T5/002 ,  G06T7/593 ,  H04N5/23229 ,  H04N5/2356

Abstract: Generating an image with a selected level of background blur includes capturing, by a first image capture device, a plurality of frames of a scene, wherein each of the plurality of frames has a different focus depth, obtaining a depth map of the scene, determining a target object and a background in the scene based on the depth map, determining a goal blur for the background, and selecting, for each pixel in an output image, a corresponding pixel from the focus stack.

Abstract translation: 生成具有所选择的背景模糊级别的图像包括由第一图像捕获装置捕获场景的多个帧，其中多个帧中的每一个具有不同的焦点深度，获得场景的深度图，确定 基于深度图的场景中的目标对象和背景，确定背景的目标模糊，以及从输出图像中的每个像素选择来自焦点堆栈的对应像素。

公开(公告)号：US20170006251A1

公开(公告)日：2017-01-05

申请号：US15266460

申请日：2016-09-15

Applicant: Apple Inc.

Inventor： Brett Keating ,  Vincent Wong ,  Todd Sachs ,  Claus Molgaard ,  Michael Rousson ,  Elliott Harris ,  Justin Titi ,  Karl Hsu ,  Jeff Brasket ,  Marco Zuliani

IPC: H04N5/77 ,  G06K9/62 ,  H04N5/232 ,  H04N5/14 ,  G06T7/00

Abstract: Systems and methods for improving automatic selection of keeper images from a commonly captured set of images are described. A combination of image type identification and image quality metrics may be used to identify one or more images in the set as keeper images. Image type identification may be used to categorize the captured images into, for example, three or more categories. The categories may include portrait, action, or “other.” Depending on the category identified, the images may be analyzed differently to identify keeper images. For portrait images, an operation may be used to identify the best set of faces. For action images, the set may be divided into sections such that keeper images selected from each section tell the story of the action. For the “other” category, the images may be analyzed such that those having higher quality metrics for an identified region of interest are selected.

Abstract translation: 描述了用于改善从一组共同拍摄的图像中保持图像的自动选择的系统和方法。 可以使用图像类型识别和图像质量度量的组合来识别该集合中的一个或多个图像作为保持器图像。 可以使用图像类型识别来将捕获的图像分类为例如三个或更多个类别。 类别可以包括纵向，动作或“其他”。根据所识别的类别，可以不同地分析图像以识别守护者图像。 对于肖像图像，可以使用操作来识别最佳面部组。 对于动作图像，该集合可以被划分为部分，使得从每个部分中选择的守护者图像讲述动作的故事。 对于“其他”类别，可以分析图像，以便选择对于所识别的感兴趣区域具有较高质量度量的图像。

公开(公告)号：US09525804B2

公开(公告)日：2016-12-20

申请号：US14474100

申请日：2014-08-30

Applicant: Apple Inc.

Inventor： Farhan A. Baqai ,  Claus Molgaard ,  Fabio Riccardi ,  Russell Pflughaupt

IPC: H04N9/077 ,  H04N5/217 ,  G06T5/00 ,  H04N9/64 ,  G06T5/10 ,  H04N5/21

CPC classification number: H04N5/217 ,  G06T5/002 ,  G06T5/10 ,  G06T2207/10024 ,  G06T2207/20016 ,  G06T2207/20048 ,  H04N5/21 ,  H04N9/077 ,  H04N9/646

Abstract: Image enhancement is achieved by separating image signals, e.g. YCbCr image signals, into a series of frequency bands and performing noise reduction on bands below a given frequency but not on bands above that frequency. The bands are summed to develop the image enhanced signals. The YCbCr, multi-band approach to denoising is able to operate independently—and in an optimized fashion—on both luma and chroma channels. Noise reduction is done based on models developed for both luma and chroma channels by measurements taken for multiple frequency bands, in multiple patches on the ColorChecker chart, and at multiple gain levels, in order to develop a simple yet robust set of models that may be tuned off-line a single time for each camera and then applied to images taken by such cameras in real-time without excessive processing requirements and with satisfactory results across illuminant types and lighting conditions.

Abstract translation: 图像增强通过分离图像信号来实现。 YCbCr图像信号转换成一系列频带，并在低于给定频率但不超过该频率的频带上进行降噪。 将频带相加以开发图像增强信号。 YCbCr多频去噪方法能够在亮度和色度通道上独立运行，并以优化方式运行。 基于通过对ColorChecker图表上的多个频段进行测量，在ColorChecker图表上以多个增益水平测量的亮度和色度通道开发的模型以及多个增益水平来进行降噪，以便开发一个简单但可靠的模型集 为每台摄像机单独调整离线时间，然后实时应用于摄像机所拍摄的图像，而无需过多的处理要求，并且在各种照明类型和照明条件下都能获得令人满意的结果。

公开(公告)号：US20160358634A1

公开(公告)日：2016-12-08

申请号：US14865752

申请日：2015-09-25

Applicant: Apple Inc.

Inventor： Claus Molgaard ,  Brett M. Keating ,  George E. Williams ,  Marco Zuliani ,  Vincent Y. Wong ,  Frank Doepke ,  Ethan J. Tira-Thompson

IPC: G11B27/34 ,  H04N5/235 ,  G06K9/46 ,  G06K9/00 ,  H04N5/232

CPC classification number: G11B27/34 ,  G06K9/00268 ,  G06K9/00718 ,  G06K9/4661 ,  H04N5/23229 ,  H04N5/23245 ,  H04N5/2352 ,  H04N5/2353 ,  H04N5/2621

Abstract: The invention relates to systems, methods, and computer readable media for responding to a user snapshot request by capturing anticipatory pre-snapshot image data as well as post-snapshot image data. The captured information may be used, depending upon the embodiment, to create archival image information and image presentation information that is both useful and pleasing to a user. The captured information may automatically be trimmed or edited to facilitate creating an enhanced image, such as a moving still image. Varying embodiments of the invention offer techniques for trimming and editing based upon the following: exposure, brightness, focus, white balance, detected motion of the camera, substantive image analysis, detected sound, image metadata, and/or any combination of the foregoing.

Abstract translation: 本发明涉及通过捕获预先预先快照图像数据以及快照后图像数据来响应用户快照请求的系统，方法和计算机可读介质。 根据实施例，可以使用捕获的信息来创建对用户有用和令人愉快的档案图像信息和图像呈现信息。 捕获的信息可以自动修剪或编辑，以便于创建增强图像，例如移动的静止图像。 本发明的不同实施例提供了基于以下内容进行修剪和编辑的技术：曝光，亮度，焦点，白平衡，检测到的相机运动，实质图像分析，检测到的声音，图像元数据和/或前述的任何组合。

公开(公告)号：US09491360B2

公开(公告)日：2016-11-08

申请号：US13911873

申请日：2013-06-06

Applicant: Apple Inc.

Inventor： Anita Nariani Schulze ,  Rolf Toft ,  Paul M. Hubel ,  Marius Tico ,  Jianping Zhou ,  Ralph Brunner ,  Claus Molgaard

IPC: H04N5/232

CPC classification number: H04N5/23277

Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. A novel combination of image quality and commonality metrics are used to identify a reference frame from a set of commonly captured images which, when the set's other images are combined with it, results in a quality stabilized image. The disclosed image quality and commonality metrics may also be used to optimize the use of a limited amount of image buffer memory during image capture sequences that return more images that the memory may accommodate at one time. Image quality and commonality metrics may also be used to effect the combination of multiple relatively long-exposure images which, when combined with a one or more final (relatively) short-exposure images, yields images exhibiting motion-induced blurring in interesting and visually pleasing ways.

Abstract translation: 描述了用于改善图像稳定操作的系统，方法和计算机可读介质。 使用图像质量和共性度量的新颖组合来从一组共同拍摄的图像中识别参考帧，当集合的其他图像与其组合时，其导致质量稳定的图像。 公开的图像质量和通用性度量还可以用于在图像捕获序列期间优化使用有限量的图像缓冲存储器，从而可以一次返回存储器可以容纳的更多图像。 图像质量和共性度量也可用于影响多个相对较长曝光图像的组合，当与一个或多个最终（相对）短曝光图像组合时，产生显示运动诱导的模糊的图像，其在有趣和视觉上令人满意 方法。

公开(公告)号：US20160301873A1

公开(公告)日：2016-10-13

申请号：US15154659

申请日：2016-05-13

Applicant: Apple Inc.

Inventor： Claus Molgaard ,  Marius Tico ,  Rolf Toft ,  Paul M. Hubel

IPC: H04N5/232 ,  H04N5/235

CPC classification number: H04N5/23277 ,  G06T5/002 ,  G06T5/003 ,  G06T5/50 ,  G06T7/20 ,  G06T7/254 ,  G06T11/60 ,  G06T2207/10004 ,  G06T2207/10144 ,  G06T2207/20221 ,  H04N5/23229 ,  H04N5/23254 ,  H04N5/23267 ,  H04N5/2355 ,  H04N5/2356 ,  H04N5/35581 ,  H04N5/3597 ,  H04N5/91

Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short-and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.

Abstract translation: 公开了从稳定的图像捕获装置捕捉和融合场景的短曝光和长曝光图像的技术。 更具体地，所公开的技术不仅使用共同拍摄的短曝光图像和长曝光图像之间的单独像素差异，而且使用长曝光图像中的遮挡区域的空间结构（例如，长曝光图像的区域 ）呈现由于场景物体运动造成的模糊）。 用于表示长曝光图像的该特征的新型装置是“空间差异图”。空间差异图可以用于识别短和长曝光图像中用于融合的像素，并且在一个实施例中可以是 用于识别来自短曝光图像的像素以过滤融合后的图像，以减少输出图像中的视觉不连续性。

公开(公告)号：US20150348242A1

公开(公告)日：2015-12-03

申请号：US14502887

申请日：2014-09-30

Applicant: Apple Inc.

Inventor： Claus Molgaard ,  Marius Tico ,  Rolf Toft ,  Paul M. Hubel

IPC: G06T5/00 ,  H04N5/232 ,  H04N5/91 ,  G06T11/60

CPC classification number: H04N5/23277 ,  G06T5/002 ,  G06T5/003 ,  G06T5/50 ,  G06T7/20 ,  G06T7/254 ,  G06T11/60 ,  G06T2207/10004 ,  G06T2207/10144 ,  G06T2207/20221 ,  H04N5/23229 ,  H04N5/23254 ,  H04N5/23267 ,  H04N5/2355 ,  H04N5/2356 ,  H04N5/35581 ,  H04N5/3597 ,  H04N5/91

Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short- and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.

Abstract translation: 公开了从稳定的图像捕获装置捕捉和融合场景的短曝光和长曝光图像的技术。 更具体地，所公开的技术不仅使用共同拍摄的短曝光图像和长曝光图像之间的单独像素差异，而且使用长曝光图像中的遮挡区域的空间结构（例如，长曝光图像的区域 ）呈现由于场景物体运动造成的模糊）。 用于表示长曝光图像的该特征的新型装置是“空间差异图”。空间差异图可以用于识别用于融合的短曝光图像和长曝光图像中的像素，并且在一个实施例中可以是 用于识别来自短曝光图像的像素以过滤融合后的图像，以减少输出图像中的视觉不连续性。