AUTO-FOCUS ENGINE ARCHITECTURE FOR IMAGE SIGNAL PROCESSOR

    公开(公告)号:US20240397217A1

    公开(公告)日:2024-11-28

    申请号:US18669229

    申请日:2024-05-20

    Applicant: Apple Inc.

    Abstract: Embodiments relate to image signal processors (ISP) that include binner circuits that down-sample an input image. An input image may include a plurality of pixels. The output image of the binner circuit may include a reduced number of pixels. The binner circuit may include a plurality of different operation modes. In a bin mode, the binner circuit may blend a subset of input pixel values to generate an output pixel quad. In a skip mode, the binner circuit may select one of the input pixel values as the output pixel pixel. The selection may be performed randomly to avoid aliasing. In a luminance mode, the binner circuit may take a weighted average of a subset of pixel values having different colors. In a color value mode, the binner circuit may select one of the colors in a subset of pixel values as an output pixel value.

    FOVEATED DOWN SAMPLING OF IMAGE DATA
    3.
    发明公开

    公开(公告)号:US20240031540A1

    公开(公告)日:2024-01-25

    申请号:US17870456

    申请日:2022-07-21

    Applicant: Apple Inc.

    CPC classification number: H04N9/64 G06T3/40

    Abstract: A foveated down sampling (FDS) circuit for down sampling of pixels in images. The FDS circuit down samples a first subset of pixels of a same color in an image using first scaling factors to generate first down sampled pixels in a first down sampled version of the image. The FDS circuit further down samples a second subset of the first down sampled pixels of the same color using second scaling factors to generate second down sampled pixels of the same color in a second down sampled version of the image. Pixels from the first subset are arranged in a first direction, and pixels from the second subset are arranged in a second direction.

    Image fusion architecture
    4.
    发明授权

    公开(公告)号:US11798146B2

    公开(公告)日:2023-10-24

    申请号:US16987202

    申请日:2020-08-06

    Applicant: Apple Inc.

    CPC classification number: G06T5/50 G06T3/0093 G06T5/002

    Abstract: Embodiments relate to circuitry for temporal processing and image fusion. An image fusion circuit receives captured images, and generates corresponding image pyramids. The generated image pyramids are raster or tiled processed, and stored in memory. A fusion module receives a first and second image pyramids from the memory, warps the first image pyramid based upon the second image pyramid, and fuses the warped first image pyramid with the second image pyramid to generate a fused image pyramid, which may be used for further processing, and may also be stored back into the memory. Because pyramid generation occurs prior to warping and fusion, and by allowing fused image pyramids to be stored back into memory, the image fusion circuitry is configurable to implement a variety of temporal processing functions involving different image fusion combinations.

    Image pyramid generation for image keypoint detection and descriptor generation

    公开(公告)号:US11494880B2

    公开(公告)日:2022-11-08

    申请号:US17206772

    申请日:2021-03-19

    Applicant: Apple Inc.

    Abstract: Embodiments relate to generating an image pyramid for feature extraction. A pyramid image generator circuit includes a first image buffer that stores image data at a first octave, a first blur filter circuit, a first spatial filter circuit, and a first decimator circuit. The first blur filter circuit generates a first pyramid image for a first scale of the first octave by applying a first amount of smoothing to the first image data stored in the first image buffer. The first spatial filter circuit and the first decimator generate second image data of a second octave that is higher than the first octave by applying a smoothing and a decimation to the first image data stored in the first image buffer. The first spatial filter circuit begins processing the first image data before the first blur filter circuit begins to process the first image data.

    IMAGE PYRAMID GENERATION FOR IMAGE KEYPOINT DETECTION AND DESCRIPTOR GENERATION

    公开(公告)号:US20220301110A1

    公开(公告)日:2022-09-22

    申请号:US17206772

    申请日:2021-03-19

    Applicant: Apple Inc.

    Abstract: Embodiments relate to generating an image pyramid for feature extraction. A pyramid image generator circuit includes a first image buffer that stores image data at a first octave, a first blur filter circuit, a first spatial filter circuit, and a first decimator circuit. The first blur filter circuit generates a first pyramid image for a first scale of the first octave by applying a first amount of smoothing to the first image data stored in the first image buffer. The first spatial filter circuit and the first decimator generate second image data of a second octave that is higher than the first octave by applying a smoothing and a decimation to the first image data stored in the first image buffer. The first spatial filter circuit begins processing the first image data before the first blur filter circuit begins to process the first image data.

    Local image warping in image processor using homography transform function

    公开(公告)号:US11164283B1

    公开(公告)日:2021-11-02

    申请号:US16858134

    申请日:2020-04-24

    Applicant: Apple Inc.

    Abstract: A feature extractor determines reference feature locations from a portion of a reference image and corresponding feature locations from a portion of a warp image. A transform module determines a homography transform function that transforms versions of the corresponding feature locations to the reference feature locations. The homography transform function has an error below a threshold level, where the error represents a difference between the transformed corresponding feature locations and the reference feature locations. The local transform module generates transform parameters by processing the homography transform function. A warper circuit warps the portion of the warp image by at least applying the transform parameters to generate a portion of a warped image.

    LOCAL IMAGE WARPING IN IMAGE PROCESSOR USING HOMOGRAPHY TRANSFORM FUNCTION

    公开(公告)号:US20210334934A1

    公开(公告)日:2021-10-28

    申请号:US16858134

    申请日:2020-04-24

    Applicant: Apple Inc.

    Abstract: A feature extractor determines reference feature locations from a portion of a reference image and corresponding feature locations from a portion of a warp image. A transform module determines a homography transform function that transforms versions of the corresponding feature locations to the reference feature locations. The homography transform function has an error below a threshold level, where the error represents a difference between the transformed corresponding feature locations and the reference feature locations. The local transform module generates transform parameters by processing the homography transform function. A warper circuit warps the portion of the warp image by at least applying the transform parameters to generate a portion of a warped image.

    Two stage multi-scale processing of image data

    公开(公告)号:US11010870B2

    公开(公告)日:2021-05-18

    申请号:US16848287

    申请日:2020-04-14

    Applicant: Apple Inc.

    Abstract: Embodiments relate to two stage multi-scale processing of an image. A first stage processing circuitry generates an unscaled single color version of the image that undergoes noise reduction before generating a high frequency component of the unscaled single color version. A scaler generates a first downscaled version of the image comprising a plurality of color components. A second stage processing circuitry generates a plurality of sequentially downscaled images based on the first downscaled version. The second stage processing circuitry processes the first downscaled version and the downscaled images to generate a processed version of the first downscaled version. The unscaled single color high frequency component and the processed version of the first downscaled version of the image are merged to generate a processed version of the image.

    Image fusion architecture
    10.
    发明授权

    公开(公告)号:US10853927B2

    公开(公告)日:2020-12-01

    申请号:US16358094

    申请日:2019-03-19

    Applicant: Apple Inc.

    Abstract: Embodiments relate to circuitry for performing fusion of two images captured with two different exposure times to generate a fused image having a higher dynamic range. Information about first keypoints is extracted from the first image by processing pixel values of pixels in the first image. A model describing correspondence between the first image and the second image is then built by processing at least the information about first keypoints. A processed version of the first image is warped using mapping information in the model to generate a warped version of the first image spatially more aligned to the second image than to the first image. The warped version of the first image is fused with a processed version of the second image to generate the fused image.

Patent Agency Ranking