公开(公告)号：US20130329781A1

公开(公告)日：2013-12-12

申请号：US13631605

申请日：2012-09-28

Applicant: APPLE INC.

Inventor： Yeping Su ,  Hsi-Jung Wu ,  Hao Pan ,  Ke Zhang

IPC: H04N7/26

CPC classification number: H04N19/10 ,  H04N21/23424 ,  H04N21/23439 ,  H04N21/242 ,  H04N21/8456

Abstract: A vide coding server may code a common video sequence into a plurality of coded data streams, each coded data stream representing the video sequence coded using coding parameters tailored for a respective transmission bit rate. The coding may cause a set of transmission units from among the coded data streams to include coded video data from a common point of the video sequence, and a first coded frame of each transmission unit of the set to be a synchronization frame. A manifest file may be built representing an index of transmission units of the respective coded data streams. The coded data streams and manifest file may be stored by the server for delivery to a client device. During download and decode, the chunks may be decoded efficiently even when switching among streams because the first frame in each chunk is a synchronization frame.

Abstract translation: 视频编码服务器可以将公共视频序列编码为多个编码数据流，每个编码数据流表示使用针对相应传输比特率定制的编码参数进行编码的视频序列。 编码可以使得编码数据流中的一组传输单元包括来自视频序列的公共点的编码视频数据和作为同步帧的组的每个传输单元的第一编码帧。 可以构建表示相应编码数据流的传输单元的索引的清单文件。 编码数据流和清单文件可以由服务器存储以传送到客户端设备。 在下载和解码期间，即使在每个块中的第一帧是同步帧时，即使切换流之间也可以有效地解码该块。

公开(公告)号：US20130235942A1

公开(公告)日：2013-09-12

申请号：US13631428

申请日：2012-09-28

Applicant: APPLE INC.

Inventor： Yeping Su ,  Hao Pan ,  Ke Zhang ,  Douglas Scott Price ,  James Oliver Normile ,  Hsi-Jung Wu

IPC: H04N7/26

CPC classification number: H04N19/577 ,  H04N19/107 ,  H04N19/117 ,  H04N19/124 ,  H04N19/139 ,  H04N19/149 ,  H04N19/15 ,  H04N19/154 ,  H04N19/156 ,  H04N19/172 ,  H04N19/192 ,  H04N19/46 ,  H04N19/58

Abstract: Video coding systems and methods protect against banding artifacts in decoded image content. According to the method, a video coder may identify, from content of pixel blocks of a frame of video data, which pixel blocks are likely to exhibit banding artifacts from the video coding/decoding processes. The video coder may assemble regions of the frame that are likely to exhibit banding artifacts based on the identified pixel blocks' locations with respect to each other. The video coder may apply anti-banding processing to pixel blocks within one or more of the identified regions and, thereafter, may code the processed frame by a compression operation.

Abstract translation: 视频编码系统和方法保护解码图像内容中的带状伪像。 根据该方法，视频编码器可以根据视频数据帧的像素块的内容从视频编码/解码处理中识别哪些像素块可能呈现带状伪影。 视频编码器可以基于所识别的像素块相对于彼此的位置来组合可能呈现带状伪影的帧的区域。 视频编码器可以对所识别的一个或多个区域内的像素块应用反绑带处理，然后可以通过压缩操作来对经处理的帧进行编码。

公开(公告)号：US11900578B2

公开(公告)日：2024-02-13

申请号：US18167786

申请日：2023-02-10

Applicant: Apple Inc.

Inventor： Can Jin ,  Nicolas Peirre Marie Frederic Bonnier ,  Hao Pan

IPC: G06T5/20 ,  G02B27/01 ,  G06F3/01 ,  G06T19/00

CPC classification number: G06T5/20 ,  G02B27/017 ,  G06F3/013 ,  G06T19/006

Abstract: A multi-layer low-pass filter is used to filter a first frame of video data representing at least a portion of an environment of an individual. A first layer of the filter has a first filtering resolution setting for a first subset of the first frame, while a second layer of the filter has a second filtering resolution setting for a second subset. The first subset includes a data element positioned along a direction of a gaze of the individual, and the second subset of the frame surrounds the first subset. A result of the filtering is compressed and transmitted via a network to a video processing engine configured to generate a modified visual representation of the environment.

公开(公告)号：US11836885B2

公开(公告)日：2023-12-05

申请号：US17053181

申请日：2019-05-06

Applicant: Apple Inc.

Inventor： Tobias Eble ,  Ye Cong ,  Cody J. White ,  Arthur Yasheng Zhang ,  Randall Rauwendaal ,  Moinul Khan ,  Jim C. Chou ,  Hao Pan ,  Nicolas Bonnier

IPC: G06T3/00 ,  G06T3/40 ,  G06T9/00

CPC classification number: G06T3/0093 ,  G06T3/40 ,  G06T9/001

Abstract: In one implementation, a method includes receiving a warped image representing simulated reality (SR) content (e.g., to be displayed in a display space), the warped image having a plurality of pixels at respective locations uniformly spaced in a grid pattern in a warped space, wherein the plurality of pixels are respectively associated with a plurality of respective pixel values and a plurality of respective scaling factors indicating a plurality of respective resolutions at a plurality of respective locations of the SR content (e.g., in the display space). The method includes processing the warped image in the warped space based on the plurality of respective scaling factors to generate a processed warped image and transmitting the processed warped image.

公开(公告)号：US11451788B2

公开(公告)日：2022-09-20

申请号：US17252701

申请日：2019-06-27

Applicant: Apple Inc.

Inventor： Hao Pan ,  Jim C. Chou ,  Felix C. Fernandes

IPC: H04N19/146 ,  H04L29/06 ,  H04N19/14 ,  H04N19/169 ,  H04N19/176 ,  H04N19/63 ,  H04L65/60

Abstract: A video encoding system in which pixel data is decomposed into frequency bands prior to encoding. The frequency bands for a slice of a frame may be buffered so that complexity statistics may be calculated across the frequency bands prior to encoding. The statistics may then be used by a rate control component in determining quantization parameters for the frequency bands for modulating the rate in the encoder for the current slice. The quantization parameters for the frequency bands may be calculated jointly to optimize the quality of the displayed frames after decoder reconstruction and wavelet synthesis on a receiving device. Information about one or more previously processed frames may be used in combination with the statistics for a current slice in determining the quantization parameters for the current slice.

公开(公告)号：US10986345B2

公开(公告)日：2021-04-20

申请号：US16277892

申请日：2019-02-15

Applicant: Apple Inc.

Inventor： Haitao Guo ,  Hao Pan ,  Guy Cote ,  Andrew Bai

IPC: H04N11/02 ,  H04N19/14 ,  G06F3/14 ,  G09G5/02 ,  G09G5/10 ,  H04N1/64 ,  H04N5/20 ,  H04N5/235 ,  H04N19/172 ,  H04N19/102 ,  H04N19/137 ,  H04N19/177 ,  H04N19/124 ,  H04N19/154 ,  H04N19/17 ,  H04N19/98 ,  H04N21/422 ,  H04N21/431 ,  G06T5/00 ,  G09G3/20 ,  G09G5/00 ,  H04N1/60 ,  H04N5/50 ,  H04N9/64 ,  H04N9/67 ,  H04N19/136 ,  H04N19/184 ,  H04N19/186 ,  H04N19/169 ,  H04N19/44 ,  H04N19/30 ,  H04N19/33 ,  H04N19/176 ,  H04N19/182 ,  H04N19/52 ,  H04N19/86 ,  H04N19/463 ,  H04N21/4223 ,  H04N21/44 ,  H04N21/4402 ,  H04N21/442 ,  H04N21/485

Abstract: Video processing techniques and pipelines that support capture, distribution, and display of high dynamic range (HDR) image data to both HDR-enabled display devices and display devices that do not support HDR imaging. A sensor pipeline may generate standard dynamic range (SDR) data from HDR data captured by a sensor using tone mapping, for example local tone mapping. Information used to generate the SDR data may be provided to a display pipeline as metadata with the generated SDR data. If a target display does not support HDR imaging, the SDR data may be directly rendered by the display pipeline. If the target display does support HDR imaging, then an inverse mapping technique may be applied to the SDR data according to the metadata to render HDR data for display. Information used in performing color gamut mapping may also be provided in the metadata and used to recover clipped colors for display.

公开(公告)号：US20210096023A1

公开(公告)日：2021-04-01

申请号：US16586411

申请日：2019-09-27

Applicant: Apple Inc.

Inventor： Xiaohua Yang ,  Ruiqi Hu ,  Hao Pan ,  Vaidyanath Mani ,  Qiang Wang ,  Haiyan He ,  Xin Wang ,  Gui Xie ,  Jin Wook Chang

IPC: G01J1/42 ,  G06T5/00 ,  G06T7/90 ,  G06T7/11 ,  H04N1/60

Abstract: A device comprises memory, a display characterized by a display characteristic, and processors coupled to the memory. The processors execute instructions causing the processors to receive data indicative of the display characteristic, data indicative of ambient lighting, and data indicative of content characteristics for a content item; determine a tone mapping curve for the content item based on the data indicative of content characteristics; determine a first, so-called “anchor” point along the tone mapping curve; modify a first portion of the tone mapping curve below the anchor point based on the data indicative of ambient lighting; modify a second portion of the tone mapping curve above the anchor point based on the data indicative of the display characteristic; perform tone mapping for the content item based on the modified toned mapping curve to obtain a tone mapped content item; and cause the display to display the tone mapped content item.

公开(公告)号：US20160353107A1

公开(公告)日：2016-12-01

申请号：US14721903

申请日：2015-05-26

Applicant: Apple Inc.

Inventor： Jian Lou ,  Congxia Dai ,  Hao Pan ,  Xiaohua Yang

IPC: H04N19/126 ,  H04N19/184 ,  H04N19/172 ,  H04N19/182 ,  H04N19/176

CPC classification number: H04N19/176 ,  H04N19/124 ,  H04N19/136 ,  H04N19/167 ,  H04N19/182

Abstract: Methods and systems provide an adaptive quantization parameter (QP) modulation scheme for video coding and compression that is sensitive to user visual perception. In an embodiment, the method includes detecting an eye sensitive region, where a region is considered sensitive based on a noticeability of a visual effect. The method includes estimating encoding parameters for image content in the detected eye sensitive region. The method further includes encoding the detected eye sensitive region with the estimated encoding parameters. The estimating the encoding parameters may be based on, among other things, a variance, a motion vector, a DC value, an edge value, and external information such as a user command or screen content. The encoding may include storing an average or maximum sum of square differences (SSD) value for a detected eye sensitive area and adjusting a QP value based on a comparison of the SSD value to generated threshold values.

Abstract translation: 方法和系统为视频编码和压缩提供了一种对用户视觉感知敏感的自适应量化参数（QP）调制方案。 在一个实施例中，该方法包括基于视觉效果的显着性检测区域被认为是敏感的眼睛敏感区域。 该方法包括估计检测到的眼睛敏感区域中的图像内容的编码参数。 该方法还包括用所估计的编码参数来编码检测到的眼睛敏感区域。 估计编码参数可以基于方差，运动矢量，DC值，边缘值以及诸如用户命令或屏幕内容的外部信息。 编码可以包括存储检测到的眼睛敏感区域的平均差异（SSD）值的平均或最大和，并且基于SSD值与生成的阈值的比较来调整QP值。

公开(公告)号：US20240394952A1

公开(公告)日：2024-11-28

申请号：US18797340

申请日：2024-08-07

Applicant: Apple Inc.

Inventor： Arthur Y Zhang ,  Ray L. Chang ,  Timothy R. Oriol ,  Ling Su ,  Gurjeet S. Saund ,  Guy Cote ,  Jim C. Chou ,  Hao Pan ,  Tobias Eble ,  Avi Bar-Zeev ,  Sheng Zhang ,  Justin A. Hensley ,  Geoffrey Stahl

IPC: G06T15/00 ,  G06F3/01 ,  G06T3/18 ,  G06T9/00 ,  H04W76/10 ,  H04W88/08

Abstract: A mixed reality system that includes a device and a base station that communicate via a wireless connection The device may include sensors that collect information about the user's environment and about the user. The information collected by the sensors may be transmitted to the base station via the wireless connection. The base station renders frames or slices based at least in part on the sensor information received from the device, encodes the frames or slices, and transmits the compressed frames or slices to the device for decoding and display. The base station may provide more computing power than conventional stand-alone systems, and the wireless connection does not tether the device to the base station as in conventional tethered systems. The system may implement methods and apparatus to maintain a target frame rate through the wireless link and to minimize latency in frame rendering, transmittal, and display.

公开(公告)号：US20240127400A1

公开(公告)日：2024-04-18

申请号：US18539160

申请日：2023-12-13

Applicant: Apple Inc.

Inventor： Can Jin ,  Nicolas Pierre Marie Frederic Bonnier ,  Hao Pan

IPC: G06T5/20 ,  G02B27/01 ,  G06F3/01 ,  G06T19/00

CPC classification number: G06T5/20 ,  G02B27/017 ,  G06F3/013 ,  G06T19/006

Abstract: A multi-layer low-pass filter is used to filter a first frame of video data representing at least a portion of an environment of an individual. A first layer of the filter has a first filtering resolution setting for a first subset of the first frame, while a second layer of the filter has a second filtering resolution setting for a second subset. The first subset includes a data element positioned along a direction of a gaze of the individual, and the second subset of the frame surrounds the first subset. A result of the filtering is compressed and transmitted via a network to a video processing engine configured to generate a modified visual representation of the environment.