公开(公告)号：US12081769B2

公开(公告)日：2024-09-03

申请号：US17933057

申请日：2022-09-16

Applicant: Apple Inc.

Inventor： Hao Pan ,  Jim C. Chou ,  Felix C. Fernandes

IPC: H04N19/146 ,  H04L65/70 ,  H04L65/75 ,  H04N19/14 ,  H04N19/169 ,  H04N19/176 ,  H04N19/63

CPC classification number: H04N19/146 ,  H04L65/70 ,  H04L65/75 ,  H04N19/14 ,  H04N19/176 ,  H04N19/188 ,  H04N19/63

Abstract: A video encoding system in which pixel data is decomposed into frequency bands prior to encoding. The frequency bands for a slice of a frame may be buffered so that complexity statistics may be calculated across the frequency bands prior to encoding. The statistics may then be used by a rate control component in determining quantization parameters for the frequency bands for modulating the rate in the encoder for the current slice. The quantization parameters for the frequency bands may be calculated jointly to optimize the quality of the displayed frames after decoder reconstruction and wavelet synthesis on a receiving device. Information about one or more previously processed frames may be used in combination with the statistics for a current slice in determining the quantization parameters for the current slice.

公开(公告)号：US20240087080A1

公开(公告)日：2024-03-14

申请号：US18510916

申请日：2023-11-16

Applicant: Apple Inc.

Inventor： Tobias Eble ,  Ye Cong ,  Cody J. White ,  Arthur Yasheng Zhang ,  Randall Rauwendaal ,  Moinul Khan ,  Jim C. Chou ,  Hao Pan ,  Nicolas Bonnier

IPC: G06T3/00 ,  G06T3/40 ,  G06T9/00

CPC classification number: G06T3/0093 ,  G06T3/40 ,  G06T9/001

Abstract: In one implementation, a method includes receiving a warped image representing simulated reality (SR) content (e.g., to be displayed in a display space), the warped image having a plurality of pixels at respective locations uniformly spaced in a grid pattern in a warped space, wherein the plurality of pixels are respectively associated with a plurality of respective pixel values and a plurality of respective scaling factors indicating a plurality of respective resolutions at a plurality of respective locations of the SR content (e.g., in the display space). The method includes processing the warped image in the warped space based on the plurality of respective scaling factors to generate a processed warped image and transmitting the processed warped image.

公开(公告)号：US11843776B2

公开(公告)日：2023-12-12

申请号：US17843160

申请日：2022-06-17

Applicant: Apple Inc.

Inventor： Krishnakanth Rapaka ,  Munehiro Nakazato ,  Jiandong Shen ,  Ganesh G. Yadav ,  Sorin Constantin Cismas ,  Jim C. Chou ,  Hao Pan

IPC: H04N19/124 ,  H04N19/60 ,  H04N19/126

CPC classification number: H04N19/124 ,  H04N19/126 ,  H04N19/60

Abstract: In one implementation, a method of encoding an image is performed at a device including one or more processors and non-transitory memory. The method includes determining a category of a spatial portion of an image based on a relation between a plurality of thresholds associated with a plurality of quantization scaling parameters and a bit rate of the spatial portion of the image at the plurality of quantization scaling parameters. The method includes quantizing the spatial portion of the image based on the categorization.

公开(公告)号：US11830214B2

公开(公告)日：2023-11-28

申请号：US16424937

申请日：2019-05-29

Applicant: Apple Inc.

Inventor： Jeffrey S. Norris ,  Alexandre Da Veiga ,  Bruno M. Sommer ,  Ye Cong ,  Tobias Eble ,  Moinul Khan ,  Nicolas Bonnier ,  Hao Pan

IPC: G06V10/44 ,  G06V20/20 ,  G06K9/00 ,  G06T7/73 ,  G02B27/01 ,  G06V10/46 ,  G06V10/24

CPC classification number: G06T7/73 ,  G02B27/017 ,  G06V10/245 ,  G06V10/462 ,  G06V20/20 ,  G06T2207/30204

Abstract: A method includes obtaining first pass-through image data characterized by a first pose. The method includes obtaining respective pixel characterization vectors for pixels in the first pass-through image data. The method includes identifying a feature of an object within the first pass-through image data in accordance with a determination that pixel characterization vectors for the feature satisfy a feature confidence threshold. The method includes displaying the first pass-through image data and an AR display marker that corresponds to the feature. The method includes obtaining second pass-through image data characterized by a second pose. The method includes transforming the AR display marker to a position associated with the second pose in order to track the feature. The method includes displaying the second pass-through image data and maintaining display of the AR display marker that corresponds to the feature of the object based on the transformation.

公开(公告)号：US20230009707A1

公开(公告)日：2023-01-12

申请号：US17933057

申请日：2022-09-16

Applicant: Apple Inc.

Inventor： Hao Pan ,  Jim C. Chou ,  Felix C. Fernandes

IPC: H04N19/146 ,  H04N19/14 ,  H04N19/176 ,  H04N19/169 ,  H04N19/63 ,  H04L65/70 ,  H04L65/75

Abstract: A video encoding system in which pixel data is decomposed into frequency bands prior to encoding. The frequency bands for a slice of a frame may be buffered so that complexity statistics may be calculated across the frequency bands prior to encoding. The statistics may then be used by a rate control component in determining quantization parameters for the frequency bands for modulating the rate in the encoder for the current slice. The quantization parameters for the frequency bands may be calculated jointly to optimize the quality of the displayed frames after decoder reconstruction and wavelet synthesis on a receiving device. Information about one or more previously processed frames may be used in combination with the statistics for a current slice in determining the quantization parameters for the current slice.

公开(公告)号：US20210312694A1

公开(公告)日：2021-10-07

申请号：US17352080

申请日：2021-06-18

Applicant: Apple Inc.

Inventor： Arthur Y Zhang ,  Ray L. Chang ,  Timothy R. Oriol ,  Ling Su ,  Gurjeet S. Saund ,  Guy Cote ,  Jim C. Chou ,  Hao Pan ,  Tobias Eble ,  Avi Bar-Zeev ,  Sheng Zhang ,  Justin A. Hensley ,  Geoffrey Stahl

IPC: G06T15/00 ,  H04W76/10 ,  G06F3/01 ,  G06T3/00 ,  G06T9/00

Abstract: A mixed reality system that includes a device and a base station that communicate via a wireless connection The device may include sensors that collect information about the user's environment and about the user. The information collected by the sensors may be transmitted to the base station via the wireless connection. The base station renders frames or slices based at least in part on the sensor information received from the device, encodes the frames or slices, and transmits the compressed frames or slices to the device for decoding and display. The base station may provide more computing power than conventional stand-alone systems, and the wireless connection does not tether the device to the base station as in conventional tethered systems. The system may implement methods and apparatus to maintain a target frame rate through the wireless link and to minimize latency in frame rendering, transmittal, and display.

公开(公告)号：US20210227226A1

公开(公告)日：2021-07-22

申请号：US17132922

申请日：2020-12-23

Applicant: Apple Inc.

Inventor： Kenneth I. Greenebaum ,  Haitao Guo ,  Hao Pan ,  Guy Cote ,  Andrew Bai

IPC: H04N19/14 ,  G06F3/14 ,  G09G5/02 ,  G09G5/10 ,  H04N1/64 ,  H04N5/20 ,  H04N5/235 ,  H04N19/172 ,  H04N19/102 ,  H04N19/137 ,  H04N19/177 ,  H04N19/124 ,  H04N19/154 ,  H04N19/17 ,  H04N19/98 ,  H04N21/422 ,  H04N21/431 ,  G06T5/00 ,  G09G3/20 ,  G09G5/00 ,  H04N1/60 ,  H04N5/50 ,  H04N9/64 ,  H04N9/67 ,  H04N19/136 ,  H04N19/184 ,  H04N19/186 ,  H04N19/169 ,  H04N19/44 ,  H04N19/30 ,  H04N19/33 ,  H04N19/176 ,  H04N19/182 ,  H04N19/52

Abstract: Adaptive video processing for a target display panel may be implemented in or by a server/encoding pipeline. The adaptive video processing methods may obtain and take into account video content and display panel-specific information including display characteristics and environmental conditions (e.g., ambient lighting and viewer location) when processing and encoding video content to be streamed to the target display panel in an ambient setting or environment. The server-side adaptive video processing methods may use this information to adjust one or more video processing functions as applied to the video data to generate video content in the color gamut and dynamic range of the target display panel that is adapted to the display panel characteristics and ambient viewing conditions.

公开(公告)号：US20210142443A1

公开(公告)日：2021-05-13

申请号：US17053181

申请日：2019-05-06

Applicant: Apple Inc.

Inventor： Tobias Eble ,  Ye Cong ,  Cody J. White ,  Arthur Yasheng Zhang ,  Randall Rauwendaal ,  Moinul Khan ,  Jim C. Chou ,  Hao Pan ,  Nicolas Bonnier

IPC: G06T3/00 ,  G06T3/40 ,  G06T9/00

Abstract: In one implementation, a method includes receiving a warped image representing simulated reality (SR) content (e.g., to be displayed in a display space), the warped image having a plurality of pixels at respective locations uniformly spaced in a grid pattern in a warped space, wherein the plurality of pixels are respectively associated with a plurality of respective pixel values and a plurality of respective scaling factors indicating a plurality of respective resolutions at a plurality of respective locations of the SR content (e.g., in the display space). The method includes processing the warped image in the warped space based on the plurality of respective scaling factors to generate a processed warped image and transmitting the processed warped image.

公开(公告)号：US09432694B2

公开(公告)日：2016-08-30

申请号：US13631428

申请日：2012-09-28

Applicant: Apple Inc.

Inventor： Yeping Su ,  Hao Pan ,  Ke Zhang ,  Douglas Scott Price ,  James Oliver Normile ,  Hsi-Jung Wu

IPC: H04N19/577 ,  H04N19/139 ,  H04N19/172 ,  H04N19/149 ,  H04N19/15 ,  H04N19/107 ,  H04N19/117 ,  H04N19/124 ,  H04N19/154 ,  H04N19/156 ,  H04N19/192 ,  H04N19/58 ,  H04N19/46

CPC classification number: H04N19/577 ,  H04N19/107 ,  H04N19/117 ,  H04N19/124 ,  H04N19/139 ,  H04N19/149 ,  H04N19/15 ,  H04N19/154 ,  H04N19/156 ,  H04N19/172 ,  H04N19/192 ,  H04N19/46 ,  H04N19/58

Abstract: Video coding systems and methods protect against banding artifacts in decoded image content. According to the method, a video coder may identify, from content of pixel blocks of a frame of video data, which pixel blocks are likely to exhibit banding artifacts from the video coding/decoding processes. The video coder may assemble regions of the frame that are likely to exhibit banding artifacts based on the identified pixel blocks' locations with respect to each other. The video coder may apply anti-banding processing to pixel blocks within one or more of the identified regions and, thereafter, may code the processed frame by a compression operation.

Abstract translation: 视频编码系统和方法保护解码图像内容中的带状伪像。 根据该方法，视频编码器可以根据视频数据帧的像素块的内容从视频编码/解码处理中识别哪些像素块可能呈现带状伪像。 视频编码器可以基于所识别的像素块相对于彼此的位置来组合可能呈现带状伪影的帧的区域。 视频编码器可以对所识别的一个或多个区域内的像素块应用反绑带处理，然后可以通过压缩操作来对经处理的帧进行编码。

公开(公告)号：US08897580B2

公开(公告)日：2014-11-25

申请号：US13664359

申请日：2012-10-30

Applicant: Apple Inc.

Inventor： Yeping Su ,  Jiefu Zhai ,  James Oliver Normile ,  Hsi-Jung Wu ,  Hao Pan

IPC: G06K9/36

CPC classification number: G09G3/2048 ,  G09G3/2066 ,  G09G2340/06 ,  G09G2350/00

Abstract: YCbCr image data may be dithered and converted into RGB data shown on a 8-bit or other bit display. Dither methods and image processors are provided which generate the banding artifact free image data during this process. Some methods and image processors may applying a stronger dither having a same mean with a larger variance to the image data before it is converted to RGB data. Others methods and image processors may calculate a quantization or encoding error and diffuse the calculated error among one or more neighboring pixel blocks.

Abstract translation: YCbCr图像数据可以抖动并转换成8位或其他位显示器上显示的RGB数据。 提供抖动方法和图像处理器，其在该处理期间产生无带纹带图像数据。 一些方法和图像处理器可以在将图像数据转换成RGB数据之前，对图像数据应用具有相同平均值的较强抖动。 其他方法和图像处理器可以计算量化或编码误差并且在一个或多个相邻像素块之间扩展所计算的误差。