Mismatched foreign light detection and mitigation in the image fusion of a two-camera system

    公开(公告)号:US10298864B2

    公开(公告)日:2019-05-21

    申请号:US15617503

    申请日:2017-06-08

    Applicant: Apple Inc.

    Abstract: Foreign lighting effects, such as lens flare, are very common in natural images. In a two-camera-system, the two images may be fused together to generate one image of a better quality. However, there are frequently different foreign light patterns in the two images that form the image pair, e.g., due to the difference in lens design, sensor and position, etc. Directly fusing such pairs of images will result in non-photorealistic images, with composed foreign light patterns from both images from the image pair. This disclosure describes a general foreign light mitigation scheme to detect all kinds of foreign light region mismatches. The detected foreign light mismatch regions may be deemphasized or excluded in the fusion step, in order to create a fused image that keeps a natural-looking foreign light pattern that is close to what was seen by the user of an image capture device during an image capture preview mode.

    Scene motion correction in fused image systems

    公开(公告)号:US10033927B2

    公开(公告)日:2018-07-24

    申请号:US15797768

    申请日:2017-10-30

    Applicant: Apple Inc.

    Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short- and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.

    STILL IMAGE STABILIZATION/OPTICAL IMAGE STABILIZATION SYNCHRONIZATION IN MULTI-CAMERA IMAGE CAPTURE

    公开(公告)号:US20180070015A1

    公开(公告)日:2018-03-08

    申请号:US15696583

    申请日:2017-09-06

    Applicant: Apple Inc.

    Abstract: Techniques are disclosed for managing image capture and processing in a multi-camera imaging system. In such a system, a pair of cameras each may output a sequence of frames representing captured image data. The cameras' output may be synchronized to each other to cause synchronism in the image capture operations of the cameras. The system may assess image quality of frames output from the cameras and, based on the image quality, designate a pair of the frames to serve as a “reference frame pair.” Thus, one frame from the first camera and a paired frame from the second camera will be designated as the reference frame pair. The system may adjust each reference frame in the pair using other frames from their respective cameras. The reference frames also may be processed by other operations within the system, such as image fusion.

    Reference frame selection for still image stabilization
    16.
    发明授权
    Reference frame selection for still image stabilization 有权
    用于静止图像稳定的参考帧选择

    公开(公告)号:US09491360B2

    公开(公告)日:2016-11-08

    申请号:US13911873

    申请日:2013-06-06

    Applicant: Apple Inc.

    CPC classification number: H04N5/23277

    Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. A novel combination of image quality and commonality metrics are used to identify a reference frame from a set of commonly captured images which, when the set's other images are combined with it, results in a quality stabilized image. The disclosed image quality and commonality metrics may also be used to optimize the use of a limited amount of image buffer memory during image capture sequences that return more images that the memory may accommodate at one time. Image quality and commonality metrics may also be used to effect the combination of multiple relatively long-exposure images which, when combined with a one or more final (relatively) short-exposure images, yields images exhibiting motion-induced blurring in interesting and visually pleasing ways.

    Abstract translation: 描述了用于改善图像稳定操作的系统,方法和计算机可读介质。 使用图像质量和共性度量的新颖组合来从一组共同拍摄的图像中识别参考帧,当集合的其他图像与其组合时,其导致质量稳定的图像。 公开的图像质量和通用性度量还可以用于在图像捕获序列期间优化使用有限量的图像缓冲存储器,从而可以一次返回存储器可以容纳的更多图像。 图像质量和共性度量也可用于影响多个相对较长曝光图像的组合,当与一个或多个最终(相对)短曝光图像组合时,产生显示运动诱导的模糊的图像,其在有趣和视觉上令人满意 方法。

    Scene Motion Correction In Fused Image Systems
    17.
    发明申请
    Scene Motion Correction In Fused Image Systems 有权
    融合图像系统中的场景运动校正

    公开(公告)号:US20160301873A1

    公开(公告)日:2016-10-13

    申请号:US15154659

    申请日:2016-05-13

    Applicant: Apple Inc.

    Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short-and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.

    Abstract translation: 公开了从稳定的图像捕获装置捕捉和融合场景的短曝光和长曝光图像的技术。 更具体地,所公开的技术不仅使用共同拍摄的短曝光图像和长曝光图像之间的单独像素差异,而且使用长曝光图像中的遮挡区域的空间结构(例如,长曝光图像的区域 )呈现由于场景物体运动造成的模糊)。 用于表示长曝光图像的该特征的新型装置是“空间差异图”。空间差异图可以用于识别短和长曝光图像中用于融合的像素,并且在一个实施例中可以是 用于识别来自短曝光图像的像素以过滤融合后的图像,以减少输出图像中的视觉不连续性。

    Scene Motion Correction In Fused Image Systems
    18.
    发明申请
    Scene Motion Correction In Fused Image Systems 有权
    融合图像系统中的场景运动校正

    公开(公告)号:US20150348242A1

    公开(公告)日:2015-12-03

    申请号:US14502887

    申请日:2014-09-30

    Applicant: Apple Inc.

    Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short- and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.

    Abstract translation: 公开了从稳定的图像捕获装置捕捉和融合场景的短曝光和长曝光图像的技术。 更具体地,所公开的技术不仅使用共同拍摄的短曝光图像和长曝光图像之间的单独像素差异,而且使用长曝光图像中的遮挡区域的空间结构(例如,长曝光图像的区域 )呈现由于场景物体运动造成的模糊)。 用于表示长曝光图像的该特征的新型装置是“空间差异图”。空间差异图可以用于识别用于融合的短曝光图像和长曝光图像中的像素,并且在一个实施例中可以是 用于识别来自短曝光图像的像素以过滤融合后的图像,以减少输出图像中的视觉不连续性。

    Dynamic Bracketing Operations for Image Stabilization
    19.
    发明申请
    Dynamic Bracketing Operations for Image Stabilization 有权
    图像稳定的动态包围操作

    公开(公告)号:US20140267802A1

    公开(公告)日:2014-09-18

    申请号:US13909700

    申请日:2013-06-04

    Applicant: Apple Inc.

    CPC classification number: H04N5/23267 H04N5/23254 H04N5/23258 H04N5/2356

    Abstract: Techniques are disclosed for selectively capturing, retaining, and combining multiple sub-exposure images or brackets to yield a final image having diminished motion-induced blur and good noise characteristics. More specifically, after or during the capture of N brackets, the M best may be identified for combining into a single output image, (N>M). As used here, the term “best” means those brackets that exhibit the least amount of relative motion with respect to one another—with one caveat: integer pixel shifts may be preferred over sub-pixel shifts.

    Abstract translation: 公开了用于选择性地捕获,保留和组合多个次曝光图像或括号以产生具有减小的运动引起的模糊和良好噪声特性的最终图像的技术。 更具体地说,在捕获N个括号之后或期间,可以识别M最佳以组合成单个输出图像(N> M)。 如这里所使用的,术语“最佳”是指相对于彼此表现出最小量的相对运动的括号 - 一个值得注意的是:整数像素偏移可能优于子像素位移。

    Region of Interest Based Image Registration
    20.
    发明申请
    Region of Interest Based Image Registration 有权
    基于兴趣的图像注册区域

    公开(公告)号:US20140126819A1

    公开(公告)日:2014-05-08

    申请号:US13670080

    申请日:2012-11-06

    Applicant: APPLE INC.

    Abstract: Techniques for registering images based on an identified region of interest (ROI) are described. In general, the disclosed techniques identify a region of ROI within an image and assign areas within the image corresponding to those regions more importance during the registration process. More particularly, the disclosed techniques may employ user-input or image content information to identify the ROI. Once identified, features within the ROI may be given more weight or significance during registration operations than other areas of the image having high-feature content but which are not as important to the individual capturing the image.

    Abstract translation: 描述了基于所识别的感兴趣区域(ROI)来注册图像的技术。 通常,所公开的技术识别图像内的ROI的区域,并且在注册过程期间分配对应于那些区域的图像中的区域更重要。 更具体地,所公开的技术可以采用用户输入或图像内容信息来识别ROI。 一旦被识别,ROI中的特征可以在注册操作期间比具有高特征内容的图像的其他区域给予更多的权重或重要性,但对于捕获图像的个体而言并不重要。

Patent Agency Ranking