公开(公告)号：US20230009707A1

公开(公告)日：2023-01-12

申请号：US17933057

申请日：2022-09-16

Applicant: Apple Inc.

Inventor： Hao Pan ,  Jim C. Chou ,  Felix C. Fernandes

IPC: H04N19/146 ,  H04N19/14 ,  H04N19/176 ,  H04N19/169 ,  H04N19/63 ,  H04L65/70 ,  H04L65/75

Abstract: A video encoding system in which pixel data is decomposed into frequency bands prior to encoding. The frequency bands for a slice of a frame may be buffered so that complexity statistics may be calculated across the frequency bands prior to encoding. The statistics may then be used by a rate control component in determining quantization parameters for the frequency bands for modulating the rate in the encoder for the current slice. The quantization parameters for the frequency bands may be calculated jointly to optimize the quality of the displayed frames after decoder reconstruction and wavelet synthesis on a receiving device. Information about one or more previously processed frames may be used in combination with the statistics for a current slice in determining the quantization parameters for the current slice.

公开(公告)号：US20220414841A1

公开(公告)日：2022-12-29

申请号：US17356223

申请日：2021-06-23

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Jian Zhou

IPC: G06T5/00 ,  G06T3/00 ,  G06N20/00

Abstract: An electronic device may include an electronic display to display an image based on processed image data. The electronic device may also include image processing circuitry to generate the processed image data. The image processing circuitry may receive input image data corresponding to an image in a first perspective and warp the input image data from the first perspective to a second perspective, generating warped image data. Additionally, the image processing circuitry may determine one or more occluded regions in the second perspective and determine fill-data corresponding to the occluded regions. The processed image data may be generated by combining the warped image data and the fill-data.

公开(公告)号：US11321813B2

公开(公告)日：2022-05-03

申请号：US16053369

申请日：2018-08-02

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Honkai Tam ,  Yun Gong

IPC: G06T3/40 ,  G06T5/00 ,  G06T5/50 ,  G06T5/20

Abstract: An electronic device may include angle detection circuitry to receive input image data including a channel of pixel data and determine pixel statistics based on a difference between a first pixel cluster and a second pixel cluster, and the second pixel cluster is offset from the first pixel cluster at a first angle. The pixel statistics may also be based on a difference between the first pixel cluster and a third pixel cluster, wherein the third pixel cluster is offset from the first pixel cluster a second angle, different from the first angle. The angle detection circuitry may also determine a best angle based on the differences, wherein the best angle approximates an angle of uniformity in content of the image. Image processing circuitry may then modify the input image data based on the best angle by modifying pixel values of the channel of pixel data.

公开(公告)号：US20210312694A1

公开(公告)日：2021-10-07

申请号：US17352080

申请日：2021-06-18

Applicant: Apple Inc.

Inventor： Arthur Y Zhang ,  Ray L. Chang ,  Timothy R. Oriol ,  Ling Su ,  Gurjeet S. Saund ,  Guy Cote ,  Jim C. Chou ,  Hao Pan ,  Tobias Eble ,  Avi Bar-Zeev ,  Sheng Zhang ,  Justin A. Hensley ,  Geoffrey Stahl

IPC: G06T15/00 ,  H04W76/10 ,  G06F3/01 ,  G06T3/00 ,  G06T9/00

Abstract: A mixed reality system that includes a device and a base station that communicate via a wireless connection The device may include sensors that collect information about the user's environment and about the user. The information collected by the sensors may be transmitted to the base station via the wireless connection. The base station renders frames or slices based at least in part on the sensor information received from the device, encodes the frames or slices, and transmits the compressed frames or slices to the device for decoding and display. The base station may provide more computing power than conventional stand-alone systems, and the wireless connection does not tether the device to the base station as in conventional tethered systems. The system may implement methods and apparatus to maintain a target frame rate through the wireless link and to minimize latency in frame rendering, transmittal, and display.

公开(公告)号：US20210211683A1

公开(公告)日：2021-07-08

申请号：US17210478

申请日：2021-03-23

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Alexandros Tourapis

IPC: H04N19/159 ,  H04N19/154 ,  H04N19/59 ,  H04N19/86 ,  H04N19/46 ,  G06T9/00 ,  G06N3/04 ,  G06N3/08 ,  H04N19/102 ,  H04N19/117 ,  H04N19/189 ,  H04N19/132 ,  H04N19/90 ,  H04N19/136 ,  H04N19/172

Abstract: System and method for improving video encoding and/or video decoding. In embodiments, a video encoding pipeline includes a main encoding pipeline that compresses source image data corresponding with an image frame by processing the source image data based at least in part on encoding parameters to generate encoded image data. Additionally the video encoding pipeline includes a machine learning block communicatively coupled to the main encoding pipeline, in which the machine learning block analyzes content of the image frame by processing the source image data based at least in part on machine learning parameters implemented in the machine learning block when the machine learning block is enabled by the encoding parameters; and the video encoding pipeline adaptively adjusts the encoding parameters based at least in part on the content expected to be present in the image frame to facilitate improving encoding efficiency.

公开(公告)号：US11024012B2

公开(公告)日：2021-06-01

申请号：US16053360

申请日：2018-08-02

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Yun Gong

IPC: G06T5/00 ,  G06T3/40 ,  G06T5/50

Abstract: An electronic device may include scaling circuitry to scale input pixel data to a greater resolution. The directional scaling circuitry may include first interpolation circuitry to receive best mode data, including one or more angles corresponding to content of the image and interpolate first pixel values at first pixel positions diagonally offset from input pixel positions of the input pixel data based on the best mode data and input pixel values corresponding to the input pixel positions. The directional scaling circuitry may also include second interpolation circuitry to receive the best mode data and the input pixel values and interpolate second pixel values at second pixel positions horizontally or vertically offset from the input pixel positions based at least in part on the best mode data and the input pixel values.

公开(公告)号：US20210142443A1

公开(公告)日：2021-05-13

申请号：US17053181

申请日：2019-05-06

Applicant: Apple Inc.

Inventor： Tobias Eble ,  Ye Cong ,  Cody J. White ,  Arthur Yasheng Zhang ,  Randall Rauwendaal ,  Moinul Khan ,  Jim C. Chou ,  Hao Pan ,  Nicolas Bonnier

IPC: G06T3/00 ,  G06T3/40 ,  G06T9/00

Abstract: In one implementation, a method includes receiving a warped image representing simulated reality (SR) content (e.g., to be displayed in a display space), the warped image having a plurality of pixels at respective locations uniformly spaced in a grid pattern in a warped space, wherein the plurality of pixels are respectively associated with a plurality of respective pixel values and a plurality of respective scaling factors indicating a plurality of respective resolutions at a plurality of respective locations of the SR content (e.g., in the display space). The method includes processing the warped image in the warped space based on the plurality of respective scaling factors to generate a processed warped image and transmitting the processed warped image.

公开(公告)号：US10897635B2

公开(公告)日：2021-01-19

申请号：US15407116

申请日：2017-01-16

Applicant: Apple Inc.

Inventor： Jim C. Chou

IPC: H04N19/63 ,  H04N19/91 ,  H04N19/119 ,  H04N19/137 ,  H04N19/174

Abstract: A display control is configured to detect a first condition related to an image frame from source image data. The display control is also configured to compress the image frame iteratively on portions of the image frame to generate a compressed frame. The display control is configured to compress the image frame iteratively when the first condition is detected. Additionally, display control is configured to determine whether to transmit the compressed frame to memory.

公开(公告)号：US20200043140A1

公开(公告)日：2020-02-06

申请号：US16053369

申请日：2018-08-02

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Honkai Tam ,  Yun Gong

IPC: G06T5/00 ,  G06T5/50 ,  G06T5/20 ,  G06T3/40

Abstract: An electronic device may include angle detection circuitry to receive input image data including a channel of pixel data and determine pixel statistics based on a difference between a first pixel cluster and a second pixel cluster, and the second pixel cluster is offset from the first pixel cluster at a first angle. The pixel statistics may also be based on a difference between the first pixel cluster and a third pixel cluster, wherein the third pixel cluster is offset from the first pixel cluster a second angle, different from the first angle. The angle detection circuitry may also determine a best angle based on the differences, wherein the best angle approximates an angle of uniformity in content of the image. Image processing circuitry may then modify the input image data based on the best angle by modifying pixel values of the channel of pixel data

公开(公告)号：US20190075301A1

公开(公告)日：2019-03-07

申请号：US15874655

申请日：2018-01-18

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Alexandros Tourapis

IPC: H04N19/159 ,  H04N19/172

Abstract: System and method for improving video encoding and/or video decoding. In embodiments, a video encoding pipeline includes a main encoding pipeline that compresses source image data corresponding with an image frame by processing the source image data based at least in part on encoding parameters to generate encoded image data. Additionally the video encoding pipeline includes a machine learning block communicatively coupled to the main encoding pipeline, in which the machine learning block analyzes content of the image frame by processing the source image data based at least in part on machine learning parameters implemented in the machine learning block when the machine learning block is enabled by the encoding parameters; and the video encoding pipeline adaptively adjusts the encoding parameters based at least in part on the content expected to be present in the image frame to facilitate improving encoding efficiency.