Machine learning video processing systems and methods

    公开(公告)号:US11616960B2

    公开(公告)日:2023-03-28

    申请号:US17210478

    申请日:2021-03-23

    Applicant: Apple Inc.

    Abstract: System and method for improving video encoding and/or video decoding. In embodiments, a video encoding pipeline includes a main encoding pipeline that compresses source image data corresponding with an image frame by processing the source image data based at least in part on encoding parameters to generate encoded image data. Additionally the video encoding pipeline includes a machine learning block communicatively coupled to the main encoding pipeline, in which the machine learning block analyzes content of the image frame by processing the source image data based at least in part on machine learning parameters implemented in the machine learning block when the machine learning block is enabled by the encoding parameters; and the video encoding pipeline adaptively adjusts the encoding parameters based at least in part on the content expected to be present in the image frame to facilitate improving encoding efficiency.

    Stacked Image Warp Operations Systems and Methods

    公开(公告)号:US20220414840A1

    公开(公告)日:2022-12-29

    申请号:US17356219

    申请日:2021-06-23

    Applicant: Apple Inc.

    Abstract: An electronic device may include an electronic display to display an image based on compensated image data in a panel space. The electronic device may also include image processing circuitry to generate the compensated image data. Further, generating the compensated image data may include determining a first inverse mapping of a pixel grid from the panel space to a rendering space and determining a forward mapping of the pixel grid from the rendering space to the panel space based on the first inverse mapping. The forward mapping may include corrections for multiple different warp operations stacked in a single warp operation. Additionally, the image processing circuitry may apply the forward mapping to input image data to generate the compensated image data.

    Priority-based video encoding and transmission

    公开(公告)号:US11496758B2

    公开(公告)日:2022-11-08

    申请号:US17252697

    申请日:2019-06-27

    Applicant: Apple Inc.

    Abstract: A video encoding system in which pixel data is decomposed into frequency bands prior to encoding. The frequency bands are organized into blocks that are provided to a block-based encoder that encodes the blocks and passes the encoded blocks to a wireless interface that packetizes the blocks for transmittal over a wireless connection. The encoder may categorize the encoded frequency bands into multiple priority levels, and may tag each frequency block with metadata indicating the frequency band represented in the block, the priority of the frequency band, and timing information. The wireless interface may then transmit or drop packets according to the priority levels of the encoded frequency blocks in the packets and/or according to the timing information of the frequency blocks in the packets.

    NEURAL NETWORK-BASED IMAGE PROCESSING WITH ARTIFACT COMPENSATION

    公开(公告)号:US20220309616A1

    公开(公告)日:2022-09-29

    申请号:US17841387

    申请日:2022-06-15

    Applicant: Apple Inc.

    Abstract: Methods and systems include neural network-based image processing and blending circuitry to blend an output of the neural network to compensate for potential artifacts from the neural network-based image processing. The neural network(s) apply image processing to image data using one or more neural networks as processed data. Enhance circuitry enhances the image data in a scaling circuitry to generate enhanced data. Blending circuitry receives the processed image data and the enhanced data along with an image plane of the processed data. The blending circuitry also determines whether the image processing using the one or more neural networks has applied a change to the image data greater than a threshold amount. The blending circuitry then, based at least in part in response to the change being greater than the threshold amount and/or edge information of the image data, blends the processed data with the enhanced data.

    LOW-LATENCY CONTEXT SWITCH SYSTEMS AND METHODS

    公开(公告)号:US20220270567A1

    公开(公告)日:2022-08-25

    申请号:US17742251

    申请日:2022-05-11

    Applicant: Apple Inc.

    Abstract: An electronic device includes a display panel and image processing circuitry. The image processing circuitry receives input image data corresponding to an image to display on the display panel, modifies the input image data by executing a first context task (e.g., lower priority task), and receives a context switch request. The image processing circuitry also pauses modification of the input image data by pausing execution of the first context task and then switches to modifying the input image data by executing a second context task (e.g., higher priority task).

Patent Agency Ranking