公开(公告)号：US11941785B2

公开(公告)日：2024-03-26

申请号：US17331602

申请日：2021-05-26

Applicant: Apple Inc.

Inventor： Jim C Chou ,  Yun Gong

IPC: G06T5/00 ,  G06T3/40 ,  G06T3/4007 ,  G06T5/50

CPC classification number: G06T5/002 ,  G06T3/4007 ,  G06T5/50 ,  G06T2207/20192 ,  G06T2207/20216

Abstract: An electronic device may include scaling circuitry to scale input pixel data to a greater resolution. The directional scaling circuitry may include first interpolation circuitry to receive best mode data, including one or more angles corresponding to content of the image and interpolate first pixel values at first pixel positions diagonally offset from input pixel positions of the input pixel data based on the best mode data and input pixel values corresponding to the input pixel positions. The directional scaling circuitry may also include second interpolation circuitry to receive the best mode data and the input pixel values and interpolate second pixel values at second pixel positions horizontally or vertically offset from the input pixel positions based at least in part on the best mode data and the input pixel values.

公开(公告)号：US11893711B2

公开(公告)日：2024-02-06

申请号：US17841387

申请日：2022-06-15

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Yun Gong

IPC: G06T3/40 ,  G06T7/11 ,  G06T5/50 ,  G06T5/00

CPC classification number: G06T5/001 ,  G06T3/4046 ,  G06T3/4053 ,  G06T5/50 ,  G06T7/11 ,  G06T2207/20084 ,  G06T2207/20192 ,  G06T2207/20221

Abstract: Methods and systems include neural network-based image processing and blending circuitry to blend an output of the neural network to compensate for potential artifacts from the neural network-based image processing. The neural network(s) apply image processing to image data using one or more neural networks as processed data. Enhance circuitry enhances the image data in a scaling circuitry to generate enhanced data. Blending circuitry receives the processed image data and the enhanced data along with an image plane of the processed data. The blending circuitry also determines whether the image processing using the one or more neural networks has applied a change to the image data greater than a threshold amount. The blending circuitry then, based at least in part in response to the change being greater than the threshold amount and/or edge information of the image data, blends the processed data with the enhanced data.

公开(公告)号：US11748850B2

公开(公告)日：2023-09-05

申请号：US17721988

申请日：2022-04-15

Applicant: Apple Inc.

Inventor： Jim Chen Chou ,  Chenge Li ,  Yun Gong

IPC: G06T3/40 ,  G06T5/50 ,  G06N3/045

CPC classification number: G06T3/4053 ,  G06N3/045 ,  G06T3/4007 ,  G06T5/50

Abstract: Embodiments relate to a super-resolution engine that converts a lower resolution input image into a higher resolution output image. The super-resolution engine includes a directional scaler, an enhancement processor, a feature detection processor, a blending logic circuit, and a neural network. The directional scaler generates directionally scaled image data by upscaling the input image. The enhancement processor generates enhanced image data by applying an example-based enhancement, a peaking filter, or some other type of non-neural network image processing scheme to the directionally scaled image data. The feature detection processor determines features indicating properties of portions of the directionally scaled image data. The neural network generates residual values defining differences between a target result of the super-resolution enhancement and the directionally scaled image data. The blending logic circuit blends the enhanced image data with the residual values according to the features.

公开(公告)号：US20200043138A1

公开(公告)日：2020-02-06

申请号：US16053342

申请日：2018-08-02

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Haiyan He ,  Yun Gong

IPC: G06T5/00 ,  G06T3/40 ,  G06T5/20 ,  G06T7/90

Abstract: An electronic device may include enhancement circuitry to enhance high resolution image data to improve perceived quality of an image corresponding to the high resolution image data. The enhancement circuitry may include tone detection circuitry to determine one or more tones within the image and apply changes to the high resolution image data based on the one or more tones. The enhancement circuitry may also include example-based improvement circuitry to compare the high resolution image data to low resolution image data and apply changes to the high resolution image data based on differences between sections of the high resolution image data and sections of the low resolution image data. The enhancement circuitry may also include channel processing circuitry to apply the first and second changes to one or more channels of the high resolution image data.

公开(公告)号：US20200043135A1

公开(公告)日：2020-02-06

申请号：US16056346

申请日：2018-08-06

Applicant: Apple Inc.

Inventor： Jim Chen Chou ,  Chenge Li ,  Yun Gong

IPC: G06T3/40 ,  G06T5/50 ,  G06N3/04

Abstract: Embodiments relate to a super-resolution engine that converts a lower resolution input image into a higher resolution output image. The super-resolution engine includes a directional scaler, an enhancement processor, a feature detection processor, a blending logic circuit, and a neural network. The directional scaler generates directionally scaled image data by upscaling the input image. The enhancement processor generates enhanced image data by applying an example-based enhancement, a peaking filter, or some other type of non-neural network image processing scheme to the directionally scaled image data. The feature detection processor determines features indicating properties of portions of the directionally scaled image data. The neural network generates residual values defining differences between a target result of the super-resolution enhancement and the directionally scaled image data. The blending logic circuit blends the enhanced image data with the residual values according to the features.

公开(公告)号：US11321813B2

公开(公告)日：2022-05-03

申请号：US16053369

申请日：2018-08-02

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Honkai Tam ,  Yun Gong

IPC: G06T3/40 ,  G06T5/00 ,  G06T5/50 ,  G06T5/20

Abstract: An electronic device may include angle detection circuitry to receive input image data including a channel of pixel data and determine pixel statistics based on a difference between a first pixel cluster and a second pixel cluster, and the second pixel cluster is offset from the first pixel cluster at a first angle. The pixel statistics may also be based on a difference between the first pixel cluster and a third pixel cluster, wherein the third pixel cluster is offset from the first pixel cluster a second angle, different from the first angle. The angle detection circuitry may also determine a best angle based on the differences, wherein the best angle approximates an angle of uniformity in content of the image. Image processing circuitry may then modify the input image data based on the best angle by modifying pixel values of the channel of pixel data.

公开(公告)号：US11024012B2

公开(公告)日：2021-06-01

申请号：US16053360

申请日：2018-08-02

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Yun Gong

IPC: G06T5/00 ,  G06T3/40 ,  G06T5/50

Abstract: An electronic device may include scaling circuitry to scale input pixel data to a greater resolution. The directional scaling circuitry may include first interpolation circuitry to receive best mode data, including one or more angles corresponding to content of the image and interpolate first pixel values at first pixel positions diagonally offset from input pixel positions of the input pixel data based on the best mode data and input pixel values corresponding to the input pixel positions. The directional scaling circuitry may also include second interpolation circuitry to receive the best mode data and the input pixel values and interpolate second pixel values at second pixel positions horizontally or vertically offset from the input pixel positions based at least in part on the best mode data and the input pixel values.

公开(公告)号：US20200043140A1

公开(公告)日：2020-02-06

申请号：US16053369

申请日：2018-08-02

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Honkai Tam ,  Yun Gong

IPC: G06T5/00 ,  G06T5/50 ,  G06T5/20 ,  G06T3/40

Abstract: An electronic device may include angle detection circuitry to receive input image data including a channel of pixel data and determine pixel statistics based on a difference between a first pixel cluster and a second pixel cluster, and the second pixel cluster is offset from the first pixel cluster at a first angle. The pixel statistics may also be based on a difference between the first pixel cluster and a third pixel cluster, wherein the third pixel cluster is offset from the first pixel cluster a second angle, different from the first angle. The angle detection circuitry may also determine a best angle based on the differences, wherein the best angle approximates an angle of uniformity in content of the image. Image processing circuitry may then modify the input image data based on the best angle by modifying pixel values of the channel of pixel data

公开(公告)号：US20240289922A1

公开(公告)日：2024-08-29

申请号：US18430431

申请日：2024-02-01

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Yun Gong

IPC: G06T5/00 ,  G06T3/4046 ,  G06T3/4053 ,  G06T5/50 ,  G06T7/11

CPC classification number: G06T5/00 ,  G06T3/4046 ,  G06T3/4053 ,  G06T5/50 ,  G06T7/11 ,  G06T2207/20084 ,  G06T2207/20192 ,  G06T2207/20221

Abstract: Methods and systems include neural network-based image processing and blending circuitry to blend an output of the neural network to compensate for potential artifacts from the neural network-based image processing. The neural network(s) apply image processing to image data using one or more neural networks as processed data. Enhance circuitry enhances the image data in a scaling circuitry to generate enhanced data. Blending circuitry receives the processed image data and the enhanced data along with an image plane of the processed data. The blending circuitry also determines whether the image processing using the one or more neural networks has applied a change to the image data greater than a threshold amount. The blending circuitry then, based at least in part in response to the change being greater than the threshold amount and/or edge information of the image data, blends the processed data with the enhanced data.

公开(公告)号：US20240096010A1

公开(公告)日：2024-03-21

申请号：US17934138

申请日：2022-09-21

Applicant: Apple Inc.

Inventor： Yun Gong ,  Jim C. Chou ,  Guy Cote

IPC: G06T15/50

CPC classification number: G06T15/503 ,  G06T3/0093

Abstract: A device may include an electronic display to display an image frame based on blended image data and image processing circuitry to generate the blended image data by combining first image data and second image data via a blend operation. The blend operation may include receiving graphics alpha data indicative of a transparency factor to be applied to the first image data to generate a first layer of the blend operation. The blend operation may also include overlaying the first layer onto a second layer that is based on the second image data. Overlaying the first layer onto the second layer may include adding first pixels values of the first image data that include negative pixel values and are augmented by the transparency factor to second pixel values of the second image data to generate blended pixel values of the blended image data.