公开(公告)号：US11972538B2

公开(公告)日：2024-04-30

申请号：US17578127

申请日：2022-01-18

Applicant: Apple Inc.

Inventor： Sarvesh Swami ,  David R Pope ,  Sheng Lin ,  Amnon D. Silverstein

IPC: G06K9/00 ,  G06T3/4007 ,  G06T3/4015 ,  G06T5/20

CPC classification number: G06T3/4015 ,  G06T3/4007 ,  G06T5/20

Abstract: Embodiments relate to a multi-mode demosaicing circuit able to receive and demosaic image data in a different raw image formats, such as Bayer raw image format and Quad Bayer raw image format. The multi-mode demosaicing circuit demosaics Quad Bayer image data by interpolating a green channel of the image data along each of a plurality of directions, generating a gradient of the image data along each of the plurality of directions, modifying the interpolated green channels based on respective gradients to generate full-resolution green channel image data, which is combined with red and blue image data to generate the demosaiced image data. Interpolation is performed for non-green pixels based on neighboring green pixels along a specified direction, modified by a residual value based upon values of one or more nearby same-color pixels and a correlation between values of the same-color pixels and neighboring green pixels.

公开(公告)号：US11936992B2

公开(公告)日：2024-03-19

申请号：US17578055

申请日：2022-01-18

Applicant: Apple Inc.

Inventor： Sarvesh Swami ,  David R Pope ,  Sheng Lin

IPC: H04N19/00 ,  H04N9/67 ,  H04N23/80 ,  H04N23/84

CPC classification number: H04N23/843 ,  H04N9/67 ,  H04N23/80

Abstract: Embodiments relate to a multi-mode demosaicing circuit able to receive and demosaic image data in a different raw image formats, such as Bayer raw image format and Quad Bayer raw image format. The multi-mode demosaicing circuit comprises different circuitry for demosaicing different image formats that access a shared working memory. In addition, the multi-mode demosaicing circuit shares memory with a post-processing and scaling circuit configured to perform subsequent post-processing and/or scaling of the demosaiced image data, in which the operations of the post-processing and scaling circuit are modified based on the original raw image format of the demosaiced image data to use different amounts of the shared memory, to compensate for additional memory utilized by the multi-mode demosaicing circuit when demosaicing certain types of image data.

公开(公告)号：US12260511B2

公开(公告)日：2025-03-25

申请号：US17578195

申请日：2022-01-18

Applicant: Apple Inc.

Inventor： Sarvesh Swami ,  David R Pope ,  Sheng Lin

IPC: G06T3/4015 ,  G06T5/50 ,  G06T5/70

Abstract: Embodiments relate to a front-end scaler circuit configured to receive and process demosaiced image data in different modes depending on if the demosaiced image data was demosaiced from Bayer or Quad Bayer raw image data. The front-end scaler circuit shares memory with a demosaicing circuit, and is configured to perform different operations that use different amounts of the shared memory based on the original image format of the demosaiced image data being processed, to compensate for additional memory utilized by the demosaicing circuit when demosaicing certain types of image data. For example, when processing image data demosaiced from Quad Bayer image data, the front-end scaler circuit discards a portion of the chrominance component data for the received image data before performing chromatic suppression, compared to when processing image data demosaiced from Bayer image data.

公开(公告)号：US20240305901A1

公开(公告)日：2024-09-12

申请号：US18118335

申请日：2023-03-07

Applicant: Apple Inc.

Inventor： Sarvesh Swami

IPC: H04N23/84

CPC classification number: H04N23/843

Abstract: Embodiments relate to demosaicing a raw image into a full-resolution green pixels and using high-frequency components of the full-resolution green pixels to add details into red image pixels and blue image pixels. The full-resolution green pixels are generated by interpolating green pixels in the raw image using residual weights based on correlation between green pixels and non-green pixels along different directions, and then combining the green pixel values interpolated along different directions using weights derived from gradient information. The full-resolution red and blue pixels are generated by determining weights according to the correlation of interpolated red and blue pixels relative to the full-resolution green pixels, and adding the high-frequency components of the full-resolution green pixels to the interpolated red and blue pixels.

公开(公告)号：US20230232122A1

公开(公告)日：2023-07-20

申请号：US17578055

申请日：2022-01-18

Applicant: Apple Inc.

Inventor： Sarvesh Swami ,  David R. Pope ,  Sheng Lin

IPC: H04N9/04 ,  H04N9/67 ,  H04N5/232

CPC classification number: H04N9/04515 ,  H04N9/67 ,  H04N5/23229

Abstract: Embodiments relate to a multi-mode demosaicing circuit able to receive and demosaic image data in a different raw image formats, such as Bayer raw image format and Quad Bayer raw image format. The multi-mode demosaicing circuit comprises different circuitry for demosaicing different image formats that access a shared working memory. In addition, the multi-mode demosaicing circuit shares memory with a post-processing and scaling circuit configured to perform subsequent post-processing and/or scaling of the demosaiced image data, in which the operations of the post-processing and scaling circuit are modified based on the original raw image format of the demosaiced image data to use different amounts of the shared memory, to compensate for additional memory utilized by the multi-mode demosaicing circuit when demosaicing certain types of image data.

公开(公告)号：US20230230200A1

公开(公告)日：2023-07-20

申请号：US17578195

申请日：2022-01-18

Applicant: Apple Inc.

Inventor： Sarvesh Swami ,  David R. Pope ,  Sheng Lin

IPC: G06T3/40 ,  G06T5/00 ,  G06T5/50

CPC classification number: G06T3/4015 ,  G06T5/002 ,  G06T5/50 ,  G06T2207/20212

Abstract: Embodiments relate to a front-end scaler circuit configured to receive and process demosaiced image data in different modes depending on if the demosaiced image data was demosaiced from Bayer or Quad Bayer raw image data. The front-end scaler circuit shares memory with a demosaicing circuit, and is configured to perform different operations that use different amounts of the shared memory based on the original image format of the demosaiced image data being processed, to compensate for additional memory utilized by the demosaicing circuit when demosaicing certain types of image data. For example, when processing image data demosaiced from Quad Bayer image data, the front-end scaler circuit discards a portion of the chrominance component data for the received image data before performing chromatic suppression, compared to when processing image data demosaiced from Bayer image data.

公开(公告)号：US20230230199A1

公开(公告)日：2023-07-20

申请号：US17578127

申请日：2022-01-18

Applicant: Apple Inc.

Inventor： Sarvesh Swami ,  David R. Pope ,  Sheng Lin ,  Amnon D. Silverstein

IPC: G06T3/40 ,  G06T5/20

CPC classification number: G06T3/4015 ,  G06T3/4007 ,  G06T5/20

Abstract: Embodiments relate to a multi-mode demosaicing circuit able to receive and demosaic image data in a different raw image formats, such as Bayer raw image format and Quad Bayer raw image format. The multi-mode demosaicing circuit demosaics Quad Bayer image data by interpolating a green channel of the image data along each of a plurality of directions, generating a gradient of the image data along each of the plurality of directions, modifying the interpolated green channels based on respective gradients to generate full-resolution green channel image data, which is combined with red and blue image data to generate the demosaiced image data. Interpolation is performed for non-green pixels based on neighboring green pixels along a specified direction, modified by a residual value based upon values of one or more nearby same-color pixels and a correlation between values of the same-color pixels and neighboring green pixels.