公开(公告)号：US20160358340A1

公开(公告)日：2016-12-08

申请号：US14869732

申请日：2015-09-29

Applicant: Apple Inc.

Inventor： Jianping Zhou ,  Geoffrey T. Anneheim ,  Xiaoxing Li ,  Benjamin M. Olson

IPC: G06T7/20 ,  G06T7/00 ,  G06K9/52 ,  H04N5/225 ,  G06T7/60 ,  G06K9/46 ,  H04N7/18

CPC classification number: G06K9/52 ,  G06K9/3233 ,  G06K9/4642 ,  G06K9/4661 ,  G06K9/4671 ,  G06K2009/4666 ,  G06T7/248 ,  G06T2207/10016

Abstract: Systems, methods, and computer readable media to rapidly identify and track an arbitrary sized object through a temporal sequence of frames is described. The object being tracked may initially be identified via a specified or otherwise known region-of-interest (ROI). A portion of that ROI can be used to generate an initial or reference histogram and luminosity measure, metrics that may be used to identify the ROI in a subsequent frame. For a frame subsequent to the initial or reference frame, a series of putative ROIs (each having its own location and size) may be identified and the “best” of the identified ROIs selected. As used here, the term “best” simply means that the more similar two frames' histograms and luminosity measures are, the better one is with respect to the other.

Abstract translation: 描述了通过帧的时间序列快速识别和跟踪任意大小的对象的系统，方法和计算机可读介质。 被跟踪的对象最初可以通过指定的或其他已知的感兴趣区域（ROI）来识别。 该ROI的一部分可用于生成初始或参考直方图和亮度度量，可用于识别后续帧中的ROI的度量。 对于初始或参考帧之后的帧，可以识别一系列推定的ROI（每个具有其自己的位置和大小），并且选择所识别的ROI的“最佳”。 如这里所使用的，术语“最佳”仅仅意味着更相似的两帧的直方图和亮度度量是相对于另一个更好的。

公开(公告)号：US20160337589A1

公开(公告)日：2016-11-17

申请号：US15218405

申请日：2016-07-25

Applicant: Apple Inc.

Inventor： Sebastien X. Beysserie ,  Damien J. Thivent ,  Jianping Zhou ,  Rudolph van der Merwe ,  Jason Klivington ,  Xiaoxing Li ,  Anders Holtsberg ,  Martin Lindberg ,  George E. Williams

IPC: H04N5/232

CPC classification number: H04N5/23267 ,  G06T5/003 ,  G06T2207/10016 ,  G06T2207/20201 ,  H04N5/23254

Abstract: Techniques to permit a digital image capture device to stabilize a video stream in real-time (during video capture operations) are presented. In general, techniques are disclosed for stabilizing video images using an overscan region and a look-ahead technique enabled by buffering a number of video input frames before generating a first stabilized video output frame. (Capturing a larger image than is displayed creates a buffer of pixels around the edge of an image; overscan is the term given to this buffer of pixels.) More particularly, techniques are disclosed for buffering an initial number of input frames so that a “current” frame can use motion data from both “past” and “future” frames to adjust the strength of a stabilization metric value so as to keep the current frame within its overscan. This look-ahead and look-behind capability permits a smoother stabilizing regime with fewer abrupt adjustments.

公开(公告)号：US09491360B2

公开(公告)日：2016-11-08

申请号：US13911873

申请日：2013-06-06

Applicant: Apple Inc.

Inventor： Anita Nariani Schulze ,  Rolf Toft ,  Paul M. Hubel ,  Marius Tico ,  Jianping Zhou ,  Ralph Brunner ,  Claus Molgaard

IPC: H04N5/232

CPC classification number: H04N5/23277

Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. A novel combination of image quality and commonality metrics are used to identify a reference frame from a set of commonly captured images which, when the set's other images are combined with it, results in a quality stabilized image. The disclosed image quality and commonality metrics may also be used to optimize the use of a limited amount of image buffer memory during image capture sequences that return more images that the memory may accommodate at one time. Image quality and commonality metrics may also be used to effect the combination of multiple relatively long-exposure images which, when combined with a one or more final (relatively) short-exposure images, yields images exhibiting motion-induced blurring in interesting and visually pleasing ways.

Abstract translation: 描述了用于改善图像稳定操作的系统，方法和计算机可读介质。 使用图像质量和共性度量的新颖组合来从一组共同拍摄的图像中识别参考帧，当集合的其他图像与其组合时，其导致质量稳定的图像。 公开的图像质量和通用性度量还可以用于在图像捕获序列期间优化使用有限量的图像缓冲存储器，从而可以一次返回存储器可以容纳的更多图像。 图像质量和共性度量也可用于影响多个相对较长曝光图像的组合，当与一个或多个最终（相对）短曝光图像组合时，产生显示运动诱导的模糊的图像，其在有趣和视觉上令人满意 方法。

公开(公告)号：US09413963B2

公开(公告)日：2016-08-09

申请号：US14467987

申请日：2014-08-25

Applicant: Apple Inc.

Inventor： Sebastien X. Beysserie ,  Damien J. Thivent ,  Jianping Zhou ,  Rudolph van der Merwe ,  Jason Klivington ,  Xiaoxing Li ,  Anders Holtsberg ,  Martin Lindberg ,  George E. Williams

IPC: H04N5/232 ,  G06T5/00 ,  G06T7/00

CPC classification number: H04N5/23267 ,  G06T5/003 ,  G06T2207/10016 ,  G06T2207/20201 ,  H04N5/23254

Abstract: Techniques to permit a digital image capture device to stabilize a video stream in real-time (during video capture operations) are presented. In general, techniques are disclosed for stabilizing video images using an overscan region and a look-ahead technique enabled by buffering a number of video input frames before generating a first stabilized video output frame. (Capturing a larger image than is displayed creates a buffer of pixels around the edge of an image; overscan is the term given to this buffer of pixels.) More particularly, techniques are disclosed for buffering an initial number of input frames so that a “current” frame can use motion data from both “past” and “future” frames to adjust the strength of a stabilization metric value so as to keep the current frame within its overscan. This look-ahead and look-behind capability permits a smoother stabilizing regime with fewer abrupt adjustments.

Abstract translation: 呈现允许数字图像捕获设备实时（视频捕获操作期间）稳定视频流的技术。 通常，公开了用于使用过扫描区域稳定视频图像的技术，以及通过在生成第一稳定视频输出帧之前缓冲多个视频输入帧而启用的先行技术。 （捕获比所显示的图像更大的图像在图像的边缘周围产生像素的缓冲器;过扫描是给予该像素缓冲器的术语。）更具体地，公开了用于缓冲​​初始数量的输入帧的技术， 当前“帧可以使用来自”过去“和”未来“帧的运动数据来调整稳定度量值的强度，以便将当前帧保持在其过扫描内。 这种前瞻性和后仰能力允许更平稳的稳定方案，具有较少的突然调整。

公开(公告)号：US09357132B2

公开(公告)日：2016-05-31

申请号：US14547107

申请日：2014-11-18

Applicant: Apple Inc.

Inventor： Damien J. Thivent ,  Rolf Toft ,  Jianping Zhou

IPC: H04N5/228 ,  H04N5/232 ,  H04N5/225

CPC classification number: H04N5/23287 ,  G02B27/646 ,  H04N5/2254 ,  H04N5/2257 ,  H04N5/23274

Abstract: In some embodiments, a method for compensating for lens motion includes estimating a starting position of a lens assembly associated with captured pixel data. The captured pixel data is captured from an image sensor. In some embodiments, the method further includes calculating from the starting position and position data received from the one or more position sensors lens movement associated with the captured pixel data. The lens movement is mapped into pixel movement associated with the captured pixel data. A transform matrix is adjusted to reflect at least the pixel movement. A limit factor associated with the position data is calculated. The captured pixel data is recalculated using the transform matrix and the limit factor.

Abstract translation: 在一些实施例中，用于补偿透镜运动的方法包括估计与捕获的像素数据相关联的透镜组件的起始位置。 从图像传感器捕获捕获的像素数据。 在一些实施例中，该方法还包括从与所捕获的像素数据相关联的一个或多个位置传感器透镜运动接收到的起始位置和位置数据进行计算。 透镜移动被映射到与捕获的像素数据相关联的像素运动。 调整变换矩阵以至少反映像素运动。 计算与位置数据相关联的极限因子。 使用变换矩阵和极限因子重新计算捕获的像素数据。

公开(公告)号：US09210317B2

公开(公告)日：2015-12-08

申请号：US14330789

申请日：2014-07-14

Applicant: Apple Inc.

Inventor： Jianping Zhou

IPC: H04N5/232 ,  G03B3/10 ,  G02B7/08 ,  G02B27/64

CPC classification number: H04N5/23212 ,  G02B7/08 ,  G02B27/64 ,  G03B3/10 ,  H04N5/23258

Abstract: A method for lens position estimation can include receiving from a lens driver a drive current value representing a current to be provided to a motor to position a camera lens of an electronic device, detecting an orientation of the electronic device using a motion sensor, determining a gravity vector based upon the orientation, and computing an estimated value of a lens position of the camera lens of the electronic device based upon the drive current value and gravity vector.

Abstract translation: 用于透镜位置估计的方法可以包括从透镜驱动器接收表示要提供给电动机的电流的驱动电流值，以定位电子设备的相机镜头，使用运动传感器检测电子设备的方位， 并且基于驱动电流值和重力矢量计算电子设备的相机透镜的透镜位置的估计值。

公开(公告)号：US20230396883A1

公开(公告)日：2023-12-07

申请号：US17805399

申请日：2022-06-03

Applicant: Apple Inc.

Inventor： Ahmed T. Kamal ,  Feng Li ,  Florian Ciurea ,  Jianping Zhou ,  Saeed Izadi ,  Sebastien X. Beysserie ,  Stephane S. Ben Soussan

IPC: H04N5/232 ,  G06V10/60 ,  G06V10/75

CPC classification number: H04N5/23267 ,  H04N5/23212 ,  H04N5/23296 ,  G06V10/60 ,  G06V10/751

Abstract: Devices, methods, and non-transitory computer readable media are disclosed herein to repair or mitigate the appearance of unwanted reflection artifacts in captured video image streams. These unwanted reflection artifacts often present themselves as brightly-colored spots, circles, rings, or halos that reflect the shape of a bright light source in the captured image. These artifacts, also referred to herein as “ghosts” or “green ghosts” (due to often having a greenish tint), are typically located in regions of the captured images where there is not actually a bright light source located in the image. In fact, such unwanted reflection artifacts often present themselves on the image sensor across the principal point of the lens from where the actual bright light source in the captured image is located. Such devices, methods and computer readable media may be configured to detect, track, and repair such unwanted reflection artifacts in an intelligent and efficient fashion.

公开(公告)号：US11158026B1

公开(公告)日：2021-10-26

申请号：US16825550

申请日：2020-03-20

Applicant: Apple Inc.

Inventor： Ahmed Tashrif Kamal ,  Florian Ciurea ,  Jianping Zhou ,  Paul Hubel

IPC: G06T3/40 ,  G06T5/00 ,  H04N13/239 ,  G06T3/00

Abstract: Electronic devices, computer readable storage media, and related methods are disclosed herein that are configured to stitch together images captured by multiple image capture devices of an image capture system. In particular, various techniques are employed to intelligently extend (and, optionally, smooth) the correspondence mapping between first and second images captured by image capture devices having different fields of view, e.g., fields of view that are at least partially overlapping and at least partially non-overlapping. The techniques may also include determining a “transitional” correspondence in a transitional region between the overlapping and non-overlapping regions of the fields of view, as well as performing one or more appearance correction operations to account for the different properties of the different image capture devices used to capture the first and second images. The techniques described herein may be employed to produce enhanced output images in either the still image or the video context.

公开(公告)号：US10148881B2

公开(公告)日：2018-12-04

申请号：US15728038

申请日：2017-10-09

Applicant: Apple Inc.

Inventor： Sebastien X. Beysserie ,  Jianping Zhou ,  Stephane S. Ben Soussan

IPC: H04N5/232

Abstract: Techniques to improve a digital image capture device's ability to stabilize a video stream—while enforcing desired stabilization constraints on particular images in the video stream—are presented that utilize an overscan region and a look-ahead technique enabled by buffering a number of video input frames before generating a first stabilized video output frame. More particularly, techniques are disclosed for buffering an initial number of input frames so that a “current” frame can use motion data from both “past” and “future” frames to adjust the value of a stabilization strength parameter and/or the weighted contribution of particular frames from the buffer in the determination of stabilization motion values for the current frame. Such techniques keep the current frame within its overscan and ensure that the stabilization constraints are enforced, while maintaining desired smoothness in the video stream. In some embodiments, the stabilization constraint may comprise a maximum allowed frame displacement.

公开(公告)号：US20170280059A1

公开(公告)日：2017-09-28

申请号：US15457377

申请日：2017-03-13

Applicant: Apple Inc.

Inventor： Damien J. Thivent ,  George E. Williams ,  Jianping Zhou ,  Richard L. Baer ,  Rolf Toft ,  Sebastien X. Beysserie

IPC: H04N5/232 ,  G06T5/20

CPC classification number: H04N5/23287 ,  G06T5/20 ,  H04N5/2254 ,  H04N5/23254 ,  H04N5/23258 ,  H04N5/23267 ,  H04N5/2328

Abstract: Techniques to improve a digital image capture device's ability to stabilize a video stream are presented. According to some embodiments, improved stabilization of captured video frames is provided by intelligently harnessing the complementary effects of both optical image stabilization (OIS) and electronic image stabilization (EIS). In particular, OIS may be used to remove intra-frame motion blur that is typically lower in amplitude and dominates with longer integration times, while EIS may be used to remove residual unwanted frame-to-frame motion that is typically larger in amplitude. The techniques disclosed herein may also leverage information provided from the image capture device's OIS system to perform improved motion blur-aware video stabilization strength modulation, which permits better video stabilization performance in low light conditions, where integration times tend to be longer, thus leading to a greater amount of motion blurring in the output stabilized video.