公开(公告)号：US11893711B2

公开(公告)日：2024-02-06

申请号：US17841387

申请日：2022-06-15

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Yun Gong

IPC: G06T3/40 ,  G06T7/11 ,  G06T5/50 ,  G06T5/00

CPC classification number: G06T5/001 ,  G06T3/4046 ,  G06T3/4053 ,  G06T5/50 ,  G06T7/11 ,  G06T2207/20084 ,  G06T2207/20192 ,  G06T2207/20221

Abstract: Methods and systems include neural network-based image processing and blending circuitry to blend an output of the neural network to compensate for potential artifacts from the neural network-based image processing. The neural network(s) apply image processing to image data using one or more neural networks as processed data. Enhance circuitry enhances the image data in a scaling circuitry to generate enhanced data. Blending circuitry receives the processed image data and the enhanced data along with an image plane of the processed data. The blending circuitry also determines whether the image processing using the one or more neural networks has applied a change to the image data greater than a threshold amount. The blending circuitry then, based at least in part in response to the change being greater than the threshold amount and/or edge information of the image data, blends the processed data with the enhanced data.

公开(公告)号：US11836885B2

公开(公告)日：2023-12-05

申请号：US17053181

申请日：2019-05-06

Applicant: Apple Inc.

Inventor： Tobias Eble ,  Ye Cong ,  Cody J. White ,  Arthur Yasheng Zhang ,  Randall Rauwendaal ,  Moinul Khan ,  Jim C. Chou ,  Hao Pan ,  Nicolas Bonnier

IPC: G06T3/00 ,  G06T3/40 ,  G06T9/00

CPC classification number: G06T3/0093 ,  G06T3/40 ,  G06T9/001

Abstract: In one implementation, a method includes receiving a warped image representing simulated reality (SR) content (e.g., to be displayed in a display space), the warped image having a plurality of pixels at respective locations uniformly spaced in a grid pattern in a warped space, wherein the plurality of pixels are respectively associated with a plurality of respective pixel values and a plurality of respective scaling factors indicating a plurality of respective resolutions at a plurality of respective locations of the SR content (e.g., in the display space). The method includes processing the warped image in the warped space based on the plurality of respective scaling factors to generate a processed warped image and transmitting the processed warped image.

公开(公告)号：US11756503B2

公开(公告)日：2023-09-12

申请号：US17742251

申请日：2022-05-11

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Honkai Tam ,  Roy G. Moss ,  Arthur L. Spence

IPC: G09G5/00 ,  G06F3/14 ,  G06T11/60 ,  G06F3/147

CPC classification number: G09G5/003 ,  G06F3/1407 ,  G06F3/147 ,  G06T11/60 ,  G09G2310/02 ,  G09G2360/18

Abstract: An electronic device includes a display panel and image processing circuitry. The image processing circuitry receives input image data corresponding to an image to display on the display panel, modifies the input image data by executing a first context task (e.g., lower priority task), and receives a context switch request. The image processing circuitry also pauses modification of the input image data by pausing execution of the first context task and then switches to modifying the input image data by executing a second context task (e.g., higher priority task).

公开(公告)号：US20230092305A1

公开(公告)日：2023-03-23

申请号：US18052807

申请日：2022-11-04

Applicant: Apple Inc.

Inventor： Sorin C. Cismas ,  Jim C. Chou ,  Ling Su ,  Keangpo R. Ho

IPC: H04N19/37 ,  H04N19/119 ,  H04N19/164 ,  H04N19/167 ,  H04N19/174 ,  H04N19/176 ,  H04N19/169 ,  H04N19/46 ,  H04N19/63

Abstract: A video encoding system in which pixel data is decomposed into frequency bands prior to encoding. The frequency bands are organized into blocks that are provided to a block-based encoder that encodes the blocks and passes the encoded blocks to a wireless interface that packetizes the blocks for transmittal over a wireless connection. The encoder may categorize the encoded frequency bands into multiple priority levels, and may tag each frequency block with metadata indicating the frequency band represented in the block, the priority of the frequency band, and timing information. The wireless interface may then transmit or drop packets according to the priority levels of the encoded frequency blocks in the packets and/or according to the timing information of the frequency blocks in the packets.

公开(公告)号：US11451788B2

公开(公告)日：2022-09-20

申请号：US17252701

申请日：2019-06-27

Applicant: Apple Inc.

Inventor： Hao Pan ,  Jim C. Chou ,  Felix C. Fernandes

IPC: H04N19/146 ,  H04L29/06 ,  H04N19/14 ,  H04N19/169 ,  H04N19/176 ,  H04N19/63 ,  H04L65/60

Abstract: A video encoding system in which pixel data is decomposed into frequency bands prior to encoding. The frequency bands for a slice of a frame may be buffered so that complexity statistics may be calculated across the frequency bands prior to encoding. The statistics may then be used by a rate control component in determining quantization parameters for the frequency bands for modulating the rate in the encoder for the current slice. The quantization parameters for the frequency bands may be calculated jointly to optimize the quality of the displayed frames after decoder reconstruction and wavelet synthesis on a receiving device. Information about one or more previously processed frames may be used in combination with the statistics for a current slice in determining the quantization parameters for the current slice.

公开(公告)号：US20210274200A1

公开(公告)日：2021-09-02

申请号：US17252697

申请日：2019-06-27

Applicant: Apple Inc.

Inventor： Sorin C. Cismas ,  Jim C. Chou ,  Ling Su ,  Keangpo R. Ho

IPC: H04N19/37 ,  H04N19/176 ,  H04N19/169 ,  H04N19/63 ,  H04N19/164 ,  H04N19/167 ,  H04N19/46 ,  H04N19/174 ,  H04N19/119

Abstract: A video encoding system in which pixel data is decomposed into frequency bands prior to encoding. The frequency bands are organized into blocks that are provided to a block-based encoder that encodes the blocks and passes the encoded blocks to a wireless interface that packetizes the blocks for transmittal over a wireless connection. The encoder may categorize the encoded frequency bands into multiple priority levels, and may tag each frequency block with metadata indicating the frequency band represented in the block, the priority of the frequency band, and timing information. The wireless interface may then transmit or drop packets according to the priority levels of the encoded frequency blocks in the packets and/or according to the timing information of the frequency blocks in the packets.

公开(公告)号：US20200043138A1

公开(公告)日：2020-02-06

申请号：US16053342

申请日：2018-08-02

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Haiyan He ,  Yun Gong

IPC: G06T5/00 ,  G06T3/40 ,  G06T5/20 ,  G06T7/90

Abstract: An electronic device may include enhancement circuitry to enhance high resolution image data to improve perceived quality of an image corresponding to the high resolution image data. The enhancement circuitry may include tone detection circuitry to determine one or more tones within the image and apply changes to the high resolution image data based on the one or more tones. The enhancement circuitry may also include example-based improvement circuitry to compare the high resolution image data to low resolution image data and apply changes to the high resolution image data based on differences between sections of the high resolution image data and sections of the low resolution image data. The enhancement circuitry may also include channel processing circuitry to apply the first and second changes to one or more channels of the high resolution image data.

公开(公告)号：US10521881B1

公开(公告)日：2019-12-31

申请号：US16015788

申请日：2018-06-22

Applicant: Apple Inc.

Inventor： Tobias Eble ,  Jim C. Chou ,  Jian Zhou ,  Moinul Khan ,  Hariprasad Puthukkootil Rajagopal

IPC: G06T3/00 ,  G02B27/01 ,  G06T7/593 ,  G06T5/00

Abstract: In various implementations, a method includes obtaining a first frame that is characterized by a first resolution associated with a first memory allocation. In some implementations, the method includes down-converting the first frame from the first resolution to a second resolution that is lower than the first resolution initially defining the first frame in order to produce a reference frame. In some implementations, the second resolution is associated with a second memory allocation that is less than a target memory allocation derived from the first memory allocation. In some implementations, the method includes storing the reference frame in a non-transitory memory. In some implementations, the method includes obtaining a second frame that is characterized by the first resolution. In some implementations, the method includes performing an error correction operation on the second frame based on the reference frame stored in the non-transitory memory.

公开(公告)号：US09807410B2

公开(公告)日：2017-10-31

申请号：US14322711

申请日：2014-07-02

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Guy Cote

IPC: H04N7/12 ,  H04N19/46 ,  H04N19/182 ,  H04N19/176 ,  H04N19/139 ,  H04N19/103 ,  H04N19/157 ,  H04N19/40 ,  H04N19/42 ,  H04N19/124 ,  H04N19/167

CPC classification number: H04N19/46 ,  H04N19/103 ,  H04N19/124 ,  H04N19/139 ,  H04N19/157 ,  H04N19/167 ,  H04N19/176 ,  H04N19/182 ,  H04N19/40 ,  H04N19/439

Abstract: Video encoders may determine an initial designation of a mode in which to encode a block of pixels in an early stage of a block processing pipeline. A component of a late stage of the block processing pipeline (one that precedes the transcoder) may determine a different mode designation for the block of pixels based on coded block pattern information, motion vector information, the position of the block in a row of such blocks, the order in which such blocks are processed in the pipeline, or other encoding related syntax elements. The component in the late stage may communicate information to the transcoder usable in coding the block of pixels, such as modified syntax elements or an end of row marker. The transcoder may encode the block of pixels in accordance with the different mode designation or may change the mode again, dependent on the communicated information.

公开(公告)号：US09571846B2

公开(公告)日：2017-02-14

申请号：US14039764

申请日：2013-09-27

Applicant: Apple Inc.

Inventor： Timothy John Millet ,  Mark P. Rygh ,  Craig M. Okruhlica ,  Jim C. Chou ,  Guy Cote ,  Gaurav S. Gulati ,  Joseph J. Cheng ,  Joseph P. Bratt

IPC: H04N19/423 ,  H04N19/56 ,  H04N19/53

CPC classification number: H04N19/423 ,  H04N19/53

Abstract: Block processing pipeline methods and apparatus in which reference data are stored to a memory according to tile formats to reduce memory accesses when fetching the data from the memory. When the pipeline stores reference data from a current frame being processed to memory as a reference frame, the reference samples are stored in macroblock sequential order. Each macroblock sample set is stored as a tile. Reference data may be stored in tile formats for luma and chroma. Chroma reference data may be stored in tile formats for chroma 4:2:0, 4:2:2, and/or 4:4:4 formats. A stage of the pipeline may write luma and chroma reference data for macroblocks to memory according to one or more of the macroblock tile formats in a modified knight's order. The stage may delay writing the reference data from the macroblocks until the macroblocks have been fully processed by the pipeline.

Abstract translation: 块处理管道方法和装置，其中参考数据根据瓦片格式存储到存储器中，以在从存储器取出数据时减少存储器访问。 当流水线将正在处理的当前帧的参考数据存储为参考帧时，参考样本以宏块顺序存储。 每个宏块样本集被存储为一个图块。 参考数据可以以瓦片和色度的瓦片格式存储。 色度参考数据可以以瓦4：2：0,4：2：2和/或4：4：4格式的瓦片格式存储。 流水线的一个阶段可以根据改进的骑士顺序中的一个或多个宏块瓦片格式将宏块的亮度和色度参考数据写入存储器。 该阶段可以延迟从宏块写入参考数据，直到宏块已被管道完全处理。