Circuit for combined down sampling and correction of image data

    公开(公告)号:US11763421B2

    公开(公告)日:2023-09-19

    申请号:US17143995

    申请日:2021-01-07

    Applicant: Apple Inc.

    CPC classification number: G06T3/4007 H04N23/88

    Abstract: A foveated down sampling and correction (FDS-C) circuit for combined down sampling and correction of chromatic aberrations in images. The FDS-C circuit performs down sampling and interpolation of pixel values of a first subset of pixels of a color in a raw image using down sampling scale factors and first interpolation coefficients to generate first corrected pixel values for pixels of the color in a first corrected version of the raw image. The FDS-C circuit further performs interpolation of pixel values of a second subset of the pixels in the first corrected version using second interpolation coefficients to generate second corrected pixel values for pixels of the color in a second corrected version of the raw image. Pixels in the first subset are arranged in a first direction, pixels in the second subset are arranged in a second direction, and the down sampling scale factors vary along the first direction.

    High dynamic range color conversion using selective interpolation for different curves

    公开(公告)号:US11252299B1

    公开(公告)日:2022-02-15

    申请号:US17176793

    申请日:2021-02-16

    Applicant: Apple Inc.

    Abstract: Embodiments relate to pixel conversion of images for display. A circuit converts input pixel values of an image using a color conversion function. The circuit is operable in different modes where each mode uses a different color conversion function. A lookup table memory circuit stores a mapping of color converted values and input pixel values according to the mode of operation where the mapping represents the color conversion function associated with the mode. The circuit produces a color converted value from the lookup table as a color converted version of a first input pixel value responsive to the first input pixel value being within a first range. The circuit may also produce a color converted version of a second input pixel value by interpolating a subset of the color converted values received from the lookup table responsive to the second input pixel being within a second input range.

    Reduction of line banding image artifacts

    公开(公告)号:US11037272B2

    公开(公告)日:2021-06-15

    申请号:US16381312

    申请日:2019-04-11

    Applicant: Apple Inc.

    Abstract: Embodiments of the present disclosure relate to reducing line banding artifacts in raw image data. If the same line of pixel sensors (e.g., in a row) in an image sensor includes a subset of pixel sensors that receive bright light and another subset of pixel sensors that receive low light, line banding artifacts may appear in the capture raw image data. To reduce or eliminate the line banding artifacts, the raw image data is normalized by adding offset values to its pixel values. The offset values are determined from the pixel values obtained from masked pixel sensors on one or both sides of the image sensor.

    Detecting keypoints in image data
    75.
    发明授权

    公开(公告)号:US10909408B2

    公开(公告)日:2021-02-02

    申请号:US15694826

    申请日:2017-09-03

    Applicant: Apple Inc.

    Inventor: David R. Pope

    Abstract: Methods and systems for detecting keypoints in image data may include an image sensor interface receiving pixel data from an image sensor. A front-end pixel data processing circuit may receive pixel data and convert the pixel data to a different color space format. A back-end pixel data processing circuit may perform one or more operations on the pixel data. An output circuit may receive pixel data and output the pixel data to a system memory. A keypoint detection circuit may receive pixel data from the image sensor interface in the image sensor pixel data format or receive pixel data after processing by the front-end or the back-end pixel data processing circuits. The keypoint detection circuit may perform a keypoint detection operation on the pixel data to detect one or more keypoints in the image frame and output to the system memory a description of the one or more keypoints.

    Directional bilateral filtering with improved noise reduction along edges

    公开(公告)号:US10825154B2

    公开(公告)日:2020-11-03

    申请号:US16358117

    申请日:2019-03-19

    Applicant: Apple Inc.

    Abstract: Embodiments relate to a bilateral filter circuit for directional filtering of an image. The directional bilateral filter circuit determines an edge direction and a weight for the edge direction by processing differences between pixel values of pixels in a first block of pixels in the image. The bilateral filter circuit determines non-directional taps for pixels in a second block by processing pixel locations, and determines directional taps by processing differences between pixel values, gradient information for the second block and the edge direction. The bilateral filter circuit determines final filter taps for pixels in the second block by blending corresponding non-directional taps and directional taps using the weight. The bilateral filter circuit obtains a pixel value of a filtered image by multiplying the final filter taps to corresponding pixel values of the pixels in the second block and adding the multiplied values.

    REDUCTION OF LINE BANDING IMAGE ARTIFACTS
    77.
    发明申请

    公开(公告)号:US20200327644A1

    公开(公告)日:2020-10-15

    申请号:US16381312

    申请日:2019-04-11

    Applicant: Apple Inc.

    Abstract: Embodiments of the present disclosure relate to reducing line banding artifacts in raw image data. If the same line of pixel sensors (e.g., in a row) in an image sensor includes a subset of pixel sensors that receive bright light and another subset of pixel sensors that receive low light, line banding artifacts may appear in the capture raw image data. To reduce or eliminate the line banding artifacts, the raw image data is normalized by adding offset values to its pixel values. The offset values are determined from the pixel values obtained from masked pixel sensors on one or both sides of the image sensor.

    AUTO-FOCUS ENGINE ARCHITECTURE FOR IMAGE SIGNAL PROCESSOR

    公开(公告)号:US20200320661A1

    公开(公告)日:2020-10-08

    申请号:US16376414

    申请日:2019-04-05

    Applicant: Apple Inc.

    Abstract: Embodiments relate to image signal processors (ISP) that include one or more auto-focus circuits. Each of the auto-focus circuits may be connected to an image sensor and may be separate from a statistics circuit and other image processing pipelines of the ISP. An image sensor may include one or more focus pixels that are used to generate data for auto-focusing. The auto-focus circuit may extract the focus pixel values and generate a signal to control the lens position of the image sensor. Each image sensor may include a separate auto-focus circuit. When other image processing pipelines of the ISP are processing the image data from one image sensor, the auto-focus circuit for another image sensor may continue to generate focus signals that control the lens position of the other image sensor. The other image sensor may be in standby but may continue to remain in focus.

    DIRECTIONAL BILATERAL FILTERING WITH IMPROVED NOISE REDUCTION ALONG EDGES

    公开(公告)号:US20200302580A1

    公开(公告)日:2020-09-24

    申请号:US16358117

    申请日:2019-03-19

    Applicant: Apple Inc.

    Abstract: Embodiments relate to a bilateral filter circuit for directional filtering of an image. The directional bilateral filter circuit determines an edge direction and a weight for the edge direction by processing differences between pixel values of pixels in a first block of pixels in the image. The bilateral filter circuit determines non-directional taps for pixels in a second block by processing pixel locations, and determines directional taps by processing differences between pixel values, gradient information for the second block and the edge direction. The bilateral filter circuit determines final filter taps for pixels in the second block by blending corresponding non-directional taps and directional taps using the weight. The bilateral filter circuit obtains a pixel value of a filtered image by multiplying the final filter taps to corresponding pixel values of the pixels in the second block and adding the multiplied values.

    Keypoint detection circuit for processing image pyramid in recursive manner

    公开(公告)号:US10769474B2

    公开(公告)日:2020-09-08

    申请号:US16101138

    申请日:2018-08-10

    Applicant: Apple Inc.

    Abstract: Embodiments relate a keypoint detection circuit for identifying keypoints in captured image frames. The keypoint detection circuit generates an image pyramid based upon a received image frame, and determine multiple sets of keypoints for each octave of the pyramid using different levels of blur. In some embodiments, the keypoint detection circuit includes multiple branches, each branch made up of one or more circuits for determining a different set of keypoints from the image, or for determining a subsampled image for a subsequent octave of the pyramid. By determining multiple sets of keypoints for each of a plurality of pyramid octaves, a larger, more varied set of keypoints can be obtained and used for object detection and matching between images.

Patent Agency Ranking