公开(公告)号：WO2016018653A1

公开(公告)日：2016-02-04

申请号：PCT/US2015/041068

申请日：2015-07-20

Applicant: APPLE INC.

Inventor： MILLS, Christopher L. ,  LIN, Sheng ,  POPE, David R. ,  SILVERSTEIN, D. Amnon ,  LIM, Suk Hwan

IPC: H04N5/262 ,  H04N5/343 ,  H04N5/347

CPC classification number: H04N5/3458 ,  G06T3/4015 ,  H04N5/2628 ,  H04N9/07 ,  H04N9/646 ,  H04N2209/046

Abstract: An input rescale module that performs cross-color correlated downscaling of sensor data in the horizontal and vertical dimensions. The module may 5 perform a first-pass demosaic of sensor data, apply horizontal and vertical scalers to resample and downsize the data in the horizontal and vertical dimensions, and then remosaic the data to provide horizontally and vertically downscaled sensor data as output for additional image processing. The module may, for example, act as a front end scaler for an image signal processor (ISP). The demosaic performed by the module may be a relatively simple demosaic, for example a demosaic function that works on 3x3 blocks of pixels. The front end of module may receive and process sensor data at two pixels per clock (ppc); the horizontal filter component reduces the sensor data down to one ppc for downstream components of the input rescale module and for the ISP pipeline.

Abstract translation: 在水平和垂直维度上执行传感器数据的交叉色相关缩小的输入重定标模块。 模块可以执行传感器数据的第一遍去马赛克，应用水平和垂直缩放器对水平和垂直尺寸的数据进行重新采样和缩小，然后重新组合数据，以提供水平和垂直的缩小的传感器数据作为附加图像的输出 处理。 该模块可以例如用作图像信号处理器（ISP）的前端缩放器。 由模块执行的去镶嵌可以是相对简单的去马赛克，例如对3×3像素块进行工作的去马赛克功能。 模块的前端可以以每个时钟两个像素（ppc）接收和处理传感器数据; 水平滤波器组件将传感器数据减少到输入重定标模块的下游组件和ISP管线的一ppc。

公开(公告)号：WO2015073124A1

公开(公告)日：2015-05-21

申请号：PCT/US2014/057599

申请日：2014-09-26

Applicant: APPLE INC.

Inventor： SILVERSTEIN, D. Amnon ,  GO, Shun Wai ,  LIM, Suk Hwan ,  MILLET, Timothy J. ,  CHEN, Ting ,  NI, Bin

IPC: H04N7/01 ,  H04N5/232

CPC classification number: H04N5/23245 ,  H04N1/212 ,  H04N5/23216 ,  H04N5/23232 ,  H04N5/23293 ,  H04N7/0122 ,  H04N9/87

Abstract: In an embodiment, an electronic device may be configured to capture still frames during video capture, but may capture the still frames in the 4x3 aspect ratio and at higher resolution than the 16x9 aspect ratio video frames. The device may interleave high resolution, 4x3 frames and lower resolution 16x9 frames in the video sequence, and may capture the nearest higher resolution, 4x3 frame when the user indicates the capture of a still frame. Alternatively, the device may display 16x9 frames in the video sequence, and then expand to 4x3 frames when a shutter button is pressed. The device may capture the still frame and return to the 16x9 video frames responsive to a release of the shutter button.

Abstract translation: 在一个实施例中，电子设备可以被配置为在视频捕获期间捕获静止帧，但是可以以4×3宽高比捕获静止帧，并且以比16×9宽高比视频帧更高的分辨率捕获静止帧。 该装置可以在视频序列中交错高分辨率，4×3帧和较低分辨率的16×9帧，并且当用户指示拍摄静止帧时可以捕获最接近的更高分辨率的4×3帧。 或者，设备可以在视频序列中显示16x9帧，然后当按下快门按钮时，扩展为4x3帧。 该装置可以捕获静止帧并响应于快门按钮的释放而返回16×9视频帧。

公开(公告)号：WO2017035440A1

公开(公告)日：2017-03-02

申请号：PCT/US2016/048892

申请日：2016-08-26

Applicant: APPLE INC.

Inventor： LIM, Suk Hwan ,  MILLS, Christopher L. ,  SILVERSTEIN, D. Amnon ,  POPE, David R. ,  LIN, Sheng

IPC: G06T1/20

CPC classification number: G06T1/20 ,  G06T1/60 ,  G06T3/4015 ,  H04N5/23229 ,  H04N5/3765 ,  H04N5/917 ,  H04N9/045

Abstract: An image processing pipeline may process image data at multiple rates. A stream of raw pixel data collected from an image sensor for an image frame may be processed through one or more pipeline stages of an image signal processor. The stream of raw pixel data may then be converted into a full-color domain and scaled to a data size that is less than an initial data size for the image frame. The converted pixel data may be processed through one or more other pipelines stages and output for storage, further processing, or display. In some embodiments, a back-end interface may be implemented as part of the image signal processor via which image data collected from sources other than the image sensor may be received and processed through various pipeline stages at the image signal processor.

Abstract translation: 图像处理流水线可以以多个速率处理图像数据。 从图像帧的图像传感器收集的原始像素数据流可以通过图像信号处理器的一个或多个流水线级进行处理。 然后可以将原始像素数据流转换为全色域并缩放为小于图像帧的初始数据大小的数据大小。 转换的像素数据可以通过一个或多个其他管道级进行处理，并输出用于存储，进一步处理或显示。 在一些实施例中，后端接口可以被实现为图像信号处理器的一部分，通过该图像信号处理器从除图像传感器之外的源收集的图像数据可以通过图像信号处理器处的各种流水线级被接收和处理。

公开(公告)号：WO2017044292A1

公开(公告)日：2017-03-16

申请号：PCT/US2016/047873

申请日：2016-08-19

Applicant: APPLE INC.

Inventor： SILVERSTEIN, D. Amnon ,  LIN, Sheng ,  POPE, David R. ,  LIM, Suk Hwan

IPC: H04N5/357 ,  G06T5/00 ,  H01L27/146 ,  H04N5/369

CPC classification number: H04N5/3572 ,  G06T5/008 ,  H01L27/14623 ,  H01L27/14643 ,  H04N5/3651 ,  H04N5/3696 ,  H04N9/646

Abstract: Systems and methods for automatic lens flare compensation may include a non-uniformity detector configured to operate on pixel data for an image in an image sensor color pattern. The non-uniformity detector may detect a non-uniformity in the pixel data in a color channel of the image sensor color pattern. The non-uniformity detector may generate output including location and magnitude values of the non-uniformity. A lens flare detector may determine, based at least on the location and magnitude values, whether the output of the non-uniformity detector corresponds to a lens flare in the image. In some embodiments, the lens flare detector may generate, in response to determining that the output corresponds to the lens flare, a representative map of the lens flare. A lens flare corrector may determine one or more pixel data correction values corresponding to the lens flare and apply the pixel data correction values to the pixel data.

Abstract translation: 用于自动镜头光斑补偿的系统和方法可以包括被配置为对图像传感器颜色图案中的图像的像素数据进行操作的非均匀性检测器。 不均匀性检测器可以检测图像传感器颜色图案的颜色通道中的像素数据的不均匀性。 不均匀性检测器可以产生包括不均匀性的位置和幅度值的输出。 透镜闪光检测器可以至少基于位置和幅度值来确定不均匀性检测器的输出是否对应于图像中的透镜闪光。 在一些实施例中，透镜光斑检测器可以响应于确定输出对应于透镜闪光而产生透镜的代表性地图闪光。 透镜闪光校正器可以确定与透镜光斑相对应的一个或多个像素数据校正值，并将像素数据校正值应用于像素数据。

公开(公告)号：WO2015038299A1

公开(公告)日：2015-03-19

申请号：PCT/US2014/051787

申请日：2014-08-20

Applicant: APPLE INC.

Inventor： LIM, Suk Hwan ,  SILVERSTEIN, D. Amnon

IPC: G06T5/00 ,  G06T5/40

CPC classification number: H04N9/646 ,  G06T5/008 ,  G06T5/40 ,  G06T2207/10016 ,  G06T2207/20012 ,  G06T2207/20016 ,  G06T2207/20021 ,  G06T2207/20208

Abstract: Image tone adjustment using local tone curve computation may be utilized to adjust luminance ranges for images. Image tone adjustment using local tone curve computation may reduce the overall contrast of an image, while maintaining local contrast in smaller areas, such as in images capturing brightly lit scenes where the difference in intensity between brightest and darkest areas is large. A desired brightness representation of the image may be generated including target luminance values for corresponding blocks of the image. For each block, one or more tone adjustment values may be computed, that when jointly applied to the respective histograms for the block and neighboring blocks results in the luminance values that match corresponding target values. The tone adjustment values may be determined by solving an under-constrained optimization problem such that optimization constraints are minimized. The image may then be adjusted according to the computed tone adjustment values.

Abstract translation: 可以使用局部色调曲线计算的图像色调调整来调整图像的亮度范围。 使用局部色调曲线计算的图像色调调整可以降低图像的整体对比度，同时保持较小区域中的局部对比度，诸如在最亮和最暗区域之间强度差异较大的明亮场景的图像中。 可以产生图像的期望的亮度表示，包括图像的相应块的目标亮度值。 对于每个块，可以计算一个或多个色调调整值，当共同应用于块和相邻块的相应直方图时，会导致与相应目标值匹配的亮度值。 音调调整值可以通过求解不受约束的优化问题来确定，使得优化约束被最小化。 然后可以根据所计算的音调调整值来调整图像。