-
21.
公开(公告)号:US11665427B2
公开(公告)日:2023-05-30
申请号:US16990432
申请日:2020-08-11
Applicant: Apple Inc.
Inventor: Paul M. Hubel , Marius Tico , Ting Chen
IPC: G06T5/00 , G06T5/50 , G06T7/00 , H04N23/68 , H04N23/80 , H04N23/741 , H04N23/951 , H04N23/45 , H04N23/71
CPC classification number: H04N23/683 , G06T5/002 , G06T5/50 , G06T7/0002 , H04N23/6811 , H04N23/6812 , H04N23/741 , H04N23/80 , H04N23/951 , G06T2207/10144 , G06T2207/20221 , H04N23/45 , H04N23/71
Abstract: Techniques are disclosed for managing image capture and processing in a multi-camera imaging system. In such a system, a pair of cameras each may output a sequence of frames representing captured image data. The cameras' output may be synchronized to each other to cause synchronism in the image capture operations of the cameras. The system may assess image quality of frames output from the cameras and, based on the image quality, designate a pair of the frames to serve as a “reference frame pair.” Thus, one frame from the first camera and a paired frame from the second camera will be designated as the reference frame pair. The system may adjust each reference frame in the pair using other frames from their respective cameras. The reference frames also may be processed by other operations within the system, such as image fusion.
-
公开(公告)号:US11562470B2
公开(公告)日:2023-01-24
申请号:US17224830
申请日:2021-04-07
Applicant: Apple Inc.
Inventor: Marius Tico , Alok Deshpande , Anita Nariani Schulze , Hao Sun , Jun Hu , Justin Michael Faryabi-Allen , Morten Poulsen , Tak Shing Wong , Wu Cheng
Abstract: Devices, methods, and computer-readable media are disclosed describing an adaptive approach for image bracket selection and fusion, e.g., to generate low noise and high dynamic range (HDR) images in a wide variety of capturing conditions. An incoming image stream may be obtained from an image capture device, wherein the incoming image stream comprises a variety of differently-exposed captures, e.g., EV0 images, EV− images, EV+ images, long exposure (or synthetic long exposure) images, EV0/EV− image pairs, etc., which are received according to a particular pattern. When a capture request is received, a set of rules and/or a decision tree may be used to evaluate one or more capture conditions associated with the images from the incoming image stream and determine which two or more images to select for a fusion operation. A noise reduction process may optionally be performed on the selected images before (or after) the registration and fusion operations.
-
公开(公告)号:US11151702B1
公开(公告)日:2021-10-19
申请号:US16564508
申请日:2019-09-09
Applicant: Apple Inc.
Inventor: Marius Tico , Tiffany Jou Cheng , Jun Hu
Abstract: Electronic devices, methods, and program storage devices for leveraging machine learning to perform improved image fusion and/or noise reduction are disclosed. An incoming image stream may be obtained from an image capture device, wherein the incoming image stream comprises a variety of differently-exposed captures, e.g., EV0 images, EV− images, EV+ images, long exposure images, EV0/EV− image pairs, etc., which are received according to a particular pattern. When a capture request is received, two or more intermediate assets may be generated based on determined combinations of images from the incoming image stream, and the intermediate assets may then be fed into a neural network that has been trained to determine one or more sets of parameters to optimally fuse and/or noise reduce the intermediate assets. In some embodiments, the network may be trained to operate on levels of pyramidal decompositions of the intermediate assets independently, for increased efficiency and memory utilization.
-
公开(公告)号:US20170171466A1
公开(公告)日:2017-06-15
申请号:US15441905
申请日:2017-02-24
Applicant: Apple Inc.
Inventor: Marius Tico , Paul M. Hubel , Frank Doepke , Todd S. Sachs
CPC classification number: H04N5/23238 , G06T5/50 , G06T7/0002 , G06T7/11 , G06T7/174 , G06T7/20 , G06T11/60 , G06T2207/20076 , G06T2207/20221 , G06T2207/30242 , H04N5/14 , H04N5/357 , H04N5/3572
Abstract: Lens flare mitigation techniques determine which pixels in images of a sequence of images are likely to be pixels affected by lens flare. Once the lens flare areas of the images are determined, unwanted lens flare effects may be mitigated by various approaches, including reducing border artifacts along a seam between successive images, discarding entire images of the sequence that contain lens flare areas, and using tone-mapping to reduce the visibility of lens flare.
-
公开(公告)号:US09357130B2
公开(公告)日:2016-05-31
申请号:US13909700
申请日:2013-06-04
Applicant: Apple Inc.
Inventor: Todd Sachs , Marius Tico , Rolf Toft , Paul Hubel , George Williams
CPC classification number: H04N5/23267 , H04N5/23254 , H04N5/23258 , H04N5/2356
Abstract: Techniques are disclosed for selectively capturing, retaining, and combining multiple sub-exposure images or brackets to yield a final image having diminished motion-induced blur and good noise characteristics. More specifically, after or during the capture of N brackets, the M best may be identified for combining into a single output image, (N>M). As used here, the term “best” means those brackets that exhibit the least amount of relative motion with respect to one another—with one caveat: integer pixel shifts may be preferred over sub-pixel shifts.
Abstract translation: 公开了用于选择性地捕获,保留和组合多个次曝光图像或括号以产生具有减小的运动引起的模糊和良好噪声特性的最终图像的技术。 更具体地说,在捕获N个括号之后或期间,可以识别M最佳以组合成单个输出图像(N> M)。 如这里所使用的,术语“最佳”是指相对于彼此表现出最小量的相对运动的括号 - 一个值得注意的是:整数像素偏移可能优于子像素位移。
-
26.
公开(公告)号:US09344638B2
公开(公告)日:2016-05-17
申请号:US14292792
申请日:2014-05-30
Applicant: Apple Inc.
Inventor: Xuemei Zhang , Marius Tico , Yingjun Bai , Farhan A. Baqai , Fabio Riccardi , Juanqing Chen , Paul M. Hubel , George E. Williams , Todd S. Sachs , Yongfang Liang
IPC: H04N5/225 , H04N5/235 , G03B7/00 , H04N3/14 , H04N5/335 , H04N5/228 , H04N5/91 , G06K9/46 , G06T5/00 , H04N5/265 , H04N5/232 , G06T5/40
CPC classification number: H04N5/2355 , G06K9/4642 , G06K9/4661 , G06T5/001 , G06T5/009 , G06T5/40 , G06T2207/10144 , G06T2207/20016 , G06T2207/20208 , G06T2207/20221 , H04N5/23245 , H04N5/265 , H04N5/91
Abstract: Systems, methods, and computer readable media to capture and process high dynamic range (HDR) images when appropriate for a scene are disclosed. When appropriate, multiple images at a single—slightly underexposed—exposure value are captured (making a constant bracket HDR capture sequence) and local tone mapping (LTM) applied to each image. Local tone map and histogram information can be used to generate a noise-amplification mask which can be used during fusion operations. Images obtained and fused in the disclosed manner provide high dynamic range with improved noise and de-ghosting characteristics.
Abstract translation: 公开了适用于场景时捕获和处理高动态范围(HDR)图像的系统,方法和计算机可读介质。 在适当的情况下,捕获单个稍微曝光不足的曝光值的多个图像(使恒定的支架HDR捕获序列)和本地色调映射(LTM)应用于每个图像。 本地色调图和直方图信息可用于生成可在融合操作期间使用的噪声放大掩模。 以所公开的方式获得和融合的图像提供高动态范围,具有改进的噪声和去重影特性。
-
公开(公告)号:US09344636B2
公开(公告)日:2016-05-17
申请号:US14292562
申请日:2014-05-30
Applicant: Apple Inc.
Inventor: Marius Tico , Rolf Toft , Paul M. Hubel
CPC classification number: H04N5/2353 , G06T5/50 , G06T2207/10144 , G06T2207/20201 , G06T2207/20221 , H04N5/23277 , H04N5/2355 , H04N5/265
Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short- and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.
Abstract translation: 公开了从稳定的图像捕获装置捕捉和融合场景的短曝光和长曝光图像的技术。 更具体地,所公开的技术不仅使用共同拍摄的短曝光图像和长曝光图像之间的单独像素差异,而且使用长曝光图像中的遮挡区域的空间结构(例如,长曝光图像的区域 )呈现由于场景物体运动造成的模糊)。 用于表示长曝光图像的该特征的新型装置是“空间差异图”。空间差异图可以用于识别用于融合的短曝光图像和长曝光图像中的像素,并且在一个实施例中可以是 用于识别来自短曝光图像的像素以过滤融合后的图像,以减少输出图像中的视觉不连续性。
-
公开(公告)号:US09262684B2
公开(公告)日:2016-02-16
申请号:US13911740
申请日:2013-06-06
Applicant: Apple Inc.
Inventor: Marius Tico , Jianping Zhou , Anita Nariani Schulze , Rolf Toft , Paul Hubel , Wei Sun
CPC classification number: G06K9/46 , G06K9/6201 , G06T5/003 , G06T5/10 , G06T2207/10016 , G06T2207/10144 , G06T2207/20016 , G06T2207/20201 , G06T2207/20208 , G06T2207/20221 , H04N5/23267
Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. Novel approaches for fusing non-reference images with a pre-selected reference frame in a set of commonly captured images are disclosed. The fusing approach may use a soft transition by using a weighted average for ghost/non-ghost pixels to avoid sudden transition between neighborhood and almost similar pixels. Additionally, the ghost/non-ghost decision can be made based on a set of neighboring pixels rather than independently for each pixel. An alternative approach may involve performing a multi-resolution decomposition of all the captured images, using temporal fusion, spatio-temporal fusion, or combinations thereof, at each level and combining the different levels to generate an output image.
Abstract translation: 描述了用于改善图像稳定操作的系统,方法和计算机可读介质。 公开了一组用于将非参考图像与一组普通捕获图像中的预选参考帧融合的新方法。 融合方法可以通过使用重影/非重像素的加权平均值来使用软转换,以避免邻域和几乎相似像素之间的突然过渡。 此外,可以基于一组相邻像素而不是针对每个像素独立地进行重影/非重影决定。 替代方法可以包括在每个级别上使用时间融合,时空融合或其组合执行所有捕获图像的多分辨率分解,并组合不同的级别以生成输出图像。
-
公开(公告)号:US20140362256A1
公开(公告)日:2014-12-11
申请号:US13911873
申请日:2013-06-06
Applicant: Apple Inc.
Inventor: Anita Nariani Schulze , Rolf Toft , Paul M. Hubel , Marius Tico , Jianping Zhou , Ralph Brunner , Claus Molgaard
IPC: H04N5/232
CPC classification number: H04N5/23277
Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. A novel combination of image quality and commonality metrics are used to identify a reference frame from a set of commonly captured images which, when the set's other images are combined with it, results in a quality stabilized image. The disclosed image quality and commonality metrics may also be used to optimize the use of a limited amount of image buffer memory during image capture sequences that return more images that the memory may accommodate at one time. Image quality and commonality metrics may also be used to effect the combination of multiple relatively long-exposure images which, when combined with a one or more final (relatively) short-exposure images, yields images exhibiting motion-induced blurring in interesting and visually pleasing ways.
Abstract translation: 描述了用于改善图像稳定操作的系统,方法和计算机可读介质。 使用图像质量和共性度量的新颖组合来从一组共同拍摄的图像中识别参考帧,当集合的其他图像与其组合时,其导致质量稳定的图像。 公开的图像质量和通用性度量还可以用于在图像捕获序列期间优化使用有限量的图像缓冲存储器,从而可以一次返回存储器可以容纳的更多图像。 图像质量和共性度量也可用于影响多个相对较长曝光图像的组合,当与一个或多个最终(相对)短曝光图像组合时,产生显示运动诱导的模糊的图像,其在有趣和视觉上令人满意 方法。
-
公开(公告)号:US20210256669A1
公开(公告)日:2021-08-19
申请号:US17224830
申请日:2021-04-07
Applicant: Apple Inc.
Inventor: Marius Tico , Alok Deshpande , Anita Nariani Schulze , Hao Sun , Jun Hu , Justin Michael Faryabi-Allen , Morten Poulsen , Tak Shing Wong , Wu Cheng
Abstract: Devices, methods, and computer-readable media are disclosed describing an adaptive approach for image bracket selection and fusion, e.g., to generate low noise and high dynamic range (HDR) images in a wide variety of capturing conditions. An incoming image stream may be obtained from an image capture device, wherein the incoming image stream comprises a variety of differently-exposed captures, e.g., EV0 images, EV− images, EV+ images, long exposure (or synthetic long exposure) images, EV0/EV− image pairs, etc., which are received according to a particular pattern. When a capture request is received, a set of rules and/or a decision tree may be used to evaluate one or more capture conditions associated with the images from the incoming image stream and determine which two or more images to select for a fusion operation. A noise reduction process may optionally be performed on the selected images before (or after) the registration and fusion operations.
-
-
-
-
-
-
-
-
-