公开(公告)号：US20220222790A1

公开(公告)日：2022-07-14

申请号：US17711808

申请日：2022-04-01

Applicant: Apple Inc.

Inventor： Can Jin ,  Nicolas Peirre Marie Frederic Bonnier ,  Hao Pan

IPC: G06T5/20 ,  G02B27/01 ,  G06F3/01 ,  G06T19/00

Abstract: A multi-layer low-pass filter is used to filter a first frame of video data representing at least a portion of an environment of an individual. A first layer of the filter has a first filtering resolution setting for a first subset of the first frame, while a second layer of the filter has a second filtering resolution setting for a second subset. The first subset includes a data element positioned along a direction of a gaze of the individual, and the second subset of the frame surrounds the first subset. A result of the filtering is compressed and transmitted via a network to a video processing engine configured to generate a modified visual representation of the environment.

公开(公告)号：US20210090225A1

公开(公告)日：2021-03-25

申请号：US17112708

申请日：2020-12-04

Applicant: Apple Inc.

Inventor： Can Jin ,  Nicolas Pierre Marie Frederic Bonnier ,  Hao Pan

IPC: G06T5/20 ,  G02B27/01 ,  G06F3/01 ,  G06T19/00

Abstract: A multi-layer low-pass filter is used to filter a first frame of video data representing at least a portion of an environment of an individual. A first layer of the filter has a first filtering resolution setting for a first subset of the first frame, while a second layer of the filter has a second filtering resolution setting for a second subset. The first subset includes a data element positioned along a direction of a gaze of the individual, and the second subset of the frame surrounds the first subset. A result of the filtering is compressed and transmitted via a network to a video processing engine configured to generate a modified visual representation of the environment.

公开(公告)号：US10602152B2

公开(公告)日：2020-03-24

申请号：US14631398

申请日：2015-02-25

Applicant: Apple Inc.

Inventor： Kenneth I. Greenebaum ,  Haitao Guo ,  Hao Pan ,  Guy Cote ,  Andrew Bai

IPC: G06F3/14 ,  G09G5/02 ,  H04N1/64 ,  H04N19/102 ,  H04N19/98 ,  H04N21/422 ,  H04N21/431 ,  G09G5/00 ,  H04N19/14 ,  G09G5/10 ,  H04N5/20 ,  H04N5/235 ,  H04N19/172 ,  H04N19/137 ,  H04N19/177 ,  H04N19/124 ,  H04N19/154 ,  H04N19/17 ,  G06T5/00 ,  G09G3/20 ,  H04N1/60 ,  H04N5/50 ,  H04N9/64 ,  H04N9/67 ,  H04N19/136 ,  H04N19/184 ,  H04N19/186 ,  H04N19/169 ,  H04N19/44 ,  H04N19/30 ,  H04N19/33 ,  H04N19/176 ,  H04N19/182 ,  H04N19/52 ,  H04N19/86 ,  H04N19/463 ,  H04N21/4223 ,  H04N21/44 ,  H04N21/4402 ,  H04N21/442 ,  H04N21/485

Abstract: Adaptive video processing for a target display panel may be implemented in or by a server/encoding pipeline. The adaptive video processing methods may obtain and take into account video content and display panel-specific information including display characteristics and environmental conditions (e.g., ambient lighting and viewer location) when processing and encoding video content to be streamed to the target display panel in an ambient setting or environment. The server-side adaptive video processing methods may use this information to adjust one or more video processing functions as applied to the video data to generate video content in the color gamut and dynamic range of the target display panel that is adapted to the display panel characteristics and ambient viewing conditions.

公开(公告)号：US20190026874A1

公开(公告)日：2019-01-24

申请号：US16040496

申请日：2018-07-19

Applicant: Apple Inc.

Inventor： Can Jin ,  Nicolas Pierre Marie Frederic Bonnier ,  Hao Pan

IPC: G06T5/20 ,  G02B27/01 ,  G06T19/00 ,  G06F3/01

Abstract: A multi-layer low-pass filter is used to filter a first frame of video data representing at least a portion of an environment of an individual. A first layer of the filter has a first filtering resolution setting for a first subset of the first frame, while a second layer of the filter has a second filtering resolution setting for a second subset. The first subset includes a data element positioned along a direction of a gaze of the individual, and the second subset of the frame surrounds the first subset. A result of the filtering is compressed and transmitted via a network to a video processing engine configured to generate a modified visual representation of the environment.

公开(公告)号：US09992499B2

公开(公告)日：2018-06-05

申请号：US14190880

申请日：2014-02-26

Applicant: Apple Inc.

Inventor： Yeping Su ,  Hao Pan ,  James O. Normile ,  Hsi-Jung Wu

IPC: H04N19/152 ,  H04N19/142 ,  H04N19/87 ,  H04N21/234 ,  H04N21/24 ,  H04N21/44 ,  H04N21/2343 ,  H04N21/845

CPC classification number: H04N19/152 ,  H04N19/142 ,  H04N21/23418 ,  H04N21/23424 ,  H04N21/23439 ,  H04N21/2402 ,  H04N21/44004 ,  H04N21/44008 ,  H04N21/8456

Abstract: Systems and methods are presented for minimizing the suddenness and immediacy of changes to the video quality perceived by users due to bandwidth fluctuations and transitions between different bitrate streams. A method may include identifying an upcoming bitrate change in a bitstream and a nearest scene cut boundary from sync frame scene cut tags included in the bitstream. The method may include calculating whether waiting until the identified nearest scene cut boundary before changing the bitrate will cause the buffer to drop below a threshold. When the buffer is calculated to not drop below the threshold, the method may postpone the upcoming bitrate change until the identified nearest scene cut boundary.

公开(公告)号：US20170272780A1

公开(公告)日：2017-09-21

申请号：US15070620

申请日：2016-03-15

Applicant: Apple Inc.

Inventor： Hao Pan ,  Yingjun Bai ,  Xuemei Zhang ,  Haitao Guo ,  Munehiro Nakazato ,  Xiaohua Yang

IPC: H04N19/86 ,  H04N19/44 ,  H04N19/186 ,  H04N19/12 ,  H04N19/156

CPC classification number: H04N19/86 ,  H04N1/6058 ,  H04N19/12 ,  H04N19/156 ,  H04N19/186 ,  H04N19/44 ,  H04N19/85

Abstract: This disclosure relates to a wide gamut encoder capable of receiving a wide gamut color image in accordance with a wide gamut standard. The encoder can encode one or more wide gamut color image pixel values into portions of narrow gamut encoding elements for transmission to a video encoder. The encoder can implement an advanced extended YCC format that is backward compatible with a P3 color gamut.

公开(公告)号：US09571827B2

公开(公告)日：2017-02-14

申请号：US13631605

申请日：2012-09-28

Applicant: Apple Inc.

Inventor： Yeping Su ,  Hsi-Jung Wu ,  Hao Pan ,  Ke Zhang

IPC: H04N19/10 ,  H04N21/234 ,  H04N21/2343 ,  H04N21/242 ,  H04N21/845 ,  H04N7/12 ,  G06K9/36

CPC classification number: H04N19/10 ,  H04N21/23424 ,  H04N21/23439 ,  H04N21/242 ,  H04N21/8456

Abstract: A video coding server may code a common video sequence into a plurality of coded data streams, each coded data stream representing the video sequence coded using coding parameters tailored for a respective transmission bit rate. The coding may cause a set of transmission units from among the coded data streams to include coded video data from a common point of the video sequence, and a first coded frame of each transmission unit of the set to be a synchronization frame. A manifest file may be built representing an index of transmission units of the respective coded data streams. The coded data streams and manifest file may be stored by the server for delivery to a client device. During download and decode, the chunks may be decoded efficiently even when switching among streams because the first frame in each chunk is a synchronization frame.

Abstract translation: 视频编码服务器可以将公共视频序列编码为多个编码数据流，每个编码数据流表示使用针对相应传输比特率定制的编码参数进行编码的视频序列。 编码可以使得编码数据流中的一组传输单元包括来自视频序列的公共点的编码视频数据和作为同步帧的组的每个传输单元的第一编码帧。 可以构建表示相应编码数据流的传输单元的索引的清单文件。 编码数据流和清单文件可以由服务器存储以传送到客户端设备。 在下载和解码期间，即使在每个块中的第一帧是同步帧时，即使切换流之间也可以有效地解码该块。

公开(公告)号：US20150245044A1

公开(公告)日：2015-08-27

申请号：US14631401

申请日：2015-02-25

Applicant: Apple Inc.

Inventor： Haitao Guo ,  Hao Pan ,  Guy Cote ,  Andrew Bai

IPC: H04N19/30 ,  H04N19/169 ,  H04N19/33 ,  H04N19/186

CPC classification number: H04N19/14 ,  G06F3/1454 ,  G06T5/009 ,  G06T2207/20208 ,  G09G3/2007 ,  G09G5/005 ,  G09G5/02 ,  G09G5/10 ,  G09G2320/0261 ,  G09G2320/0271 ,  G09G2320/0613 ,  G09G2320/062 ,  G09G2320/0626 ,  G09G2320/066 ,  G09G2320/0666 ,  G09G2320/0673 ,  G09G2320/0686 ,  G09G2320/0693 ,  G09G2320/08 ,  G09G2320/103 ,  G09G2340/02 ,  G09G2360/144 ,  G09G2370/042 ,  H04N1/6066 ,  H04N1/64 ,  H04N5/20 ,  H04N5/2355 ,  H04N5/505 ,  H04N9/641 ,  H04N9/67 ,  H04N19/102 ,  H04N19/124 ,  H04N19/136 ,  H04N19/137 ,  H04N19/154 ,  H04N19/17 ,  H04N19/172 ,  H04N19/176 ,  H04N19/177 ,  H04N19/182 ,  H04N19/184 ,  H04N19/186 ,  H04N19/188 ,  H04N19/30 ,  H04N19/33 ,  H04N19/44 ,  H04N19/463 ,  H04N19/52 ,  H04N19/86 ,  H04N19/98 ,  H04N21/42202 ,  H04N21/4223 ,  H04N21/4318 ,  H04N21/44008 ,  H04N21/4402 ,  H04N21/44218 ,  H04N21/4854

Abstract: Video processing techniques and pipelines that support capture, distribution, and display of high dynamic range (HDR) image data to both HDR-enabled display devices and display devices that do not support HDR imaging. A sensor pipeline may generate standard dynamic range (SDR) data from HDR data captured by a sensor using tone mapping, for example local tone mapping. Information used to generate the SDR data may be provided to a display pipeline as metadata with the generated SDR data. If a target display does not support HDR imaging, the SDR data may be directly rendered by the display pipeline. If the target display does support HDR imaging, then an inverse mapping technique may be applied to the SDR data according to the metadata to render HDR data for display. Information used in performing color gamut mapping may also be provided in the metadata and used to recover clipped colors for display.

Abstract translation: 视频处理技术和管道支持将HDR动态范围（HDR）图像数据捕获，分配和显示到支持HDR的显示设备和不支持HDR成像的显示设备。 传感器管线可以从使用色调映射的传感器捕获的HDR数据（例如本地色调映射）生成标准动态范围（SDR）数据。 用于生成SDR数据的信息可以作为具有生成的SDR数据的元数据提供给显示管道。 如果目标显示不支持HDR成像，则SDR数据可以由显示管道直接呈现。 如果目标显示确实支持HDR成像，则可以根据元数据将逆映射技术应用于SDR数据，以呈现HDR数据进行显示。 用于执行色域映射的信息也可以在元数据中提供，并用于恢复被剪辑的颜色以供显示。

公开(公告)号：US20150243243A1

公开(公告)日：2015-08-27

申请号：US14631398

申请日：2015-02-25

Applicant: Apple Inc.

Inventor： Kenneth I. Greenebaum ,  Haitao Guo ,  Hao Pan ,  Guy Cote ,  Andrew Bai

IPC: G09G5/00 ,  G09G3/20 ,  H04N1/60 ,  H04N9/67 ,  H04N9/64 ,  G06T5/00 ,  H04N5/50

Abstract: Adaptive video processing for a target display panel may be implemented in or by a server/encoding pipeline. The adaptive video processing methods may obtain and take into account video content and display panel-specific information including display characteristics and environmental conditions (e.g., ambient lighting and viewer location) when processing and encoding video content to be streamed to the target display panel in an ambient setting or environment. The server-side adaptive video processing methods may use this information to adjust one or more video processing functions as applied to the video data to generate video content in the color gamut and dynamic range of the target display panel that is adapted to the display panel characteristics and ambient viewing conditions.

Abstract translation: 用于目标显示面板的自适应视频处理可以在服务器/编码流水线中或由服务器/编码流水线实现。 当处理和编码要流式传输到目标显示面板的视频内容时，自适应视频处理方法可以获得并考虑视频内容和显示面板特定信息，包括显示特性和环境条件（例如，环境照明和观看者位置） 环境设置或环境。 服务器侧自适应视频处理方法可以使用该信息来调整应用于视频数据的一个或多个视频处理功能，以生成适合于显示面板特征的目标显示面板的色域和动态范围中的视频内容 和环境观察条件。

公开(公告)号：US20150243200A1

公开(公告)日：2015-08-27

申请号：US14631405

申请日：2015-02-25

Applicant: APPLE INC.

Inventor： Hao Pan

IPC: G09G3/20

CPC classification number: H04N19/14 ,  G06F3/1454 ,  G06T5/009 ,  G06T2207/20208 ,  G09G3/2007 ,  G09G5/005 ,  G09G5/02 ,  G09G5/10 ,  G09G2320/0261 ,  G09G2320/0271 ,  G09G2320/0613 ,  G09G2320/062 ,  G09G2320/0626 ,  G09G2320/066 ,  G09G2320/0666 ,  G09G2320/0673 ,  G09G2320/0686 ,  G09G2320/0693 ,  G09G2320/08 ,  G09G2320/103 ,  G09G2340/02 ,  G09G2360/144 ,  G09G2370/042 ,  H04N1/6066 ,  H04N1/64 ,  H04N5/20 ,  H04N5/2355 ,  H04N5/505 ,  H04N9/641 ,  H04N9/67 ,  H04N19/102 ,  H04N19/124 ,  H04N19/136 ,  H04N19/137 ,  H04N19/154 ,  H04N19/17 ,  H04N19/172 ,  H04N19/176 ,  H04N19/177 ,  H04N19/182 ,  H04N19/184 ,  H04N19/186 ,  H04N19/188 ,  H04N19/30 ,  H04N19/33 ,  H04N19/44 ,  H04N19/463 ,  H04N19/52 ,  H04N19/86 ,  H04N19/98 ,  H04N21/42202 ,  H04N21/4223 ,  H04N21/4318 ,  H04N21/44008 ,  H04N21/4402 ,  H04N21/44218 ,  H04N21/4854

Abstract: Display brightness adjustment apparatus and methods are described in which the average brightness of a display may be scaled up or down using a non-linear function. When applying the non-linear function to scale down brightness, the contrast of the output signal may not be reduced so that the dynamic range and highlights are preserved. The non-linear brightness adjustment may be performed automatically, for example in response to ambient light level as detected by sensor(s), but may also be applied in response to a user adjustment to a brightness control knob or slider. The non-linear brightness adjustment may be performed globally, or alternatively may be performed on local regions of an image or display panel. The non-linear function may be a piecewise linear function, or some other non-linear function.

Abstract translation: 描述显示亮度调节装置和方法，其中可以使用非线性函数来将显示器的平均亮度按比例放大或缩小。 当应用非线性函数缩小亮度时，输出信号的对比度可能不会降低，从而保留动态范围和高光。 可以例如响应于由传感器检测到的环境光水平而自动执行非线性亮度调节，但是也可以响应于对亮度控制旋钮或滑块的用户调整而应用非线性亮度调节。 可以全局地执行非线性亮度调整，或者可以在图像或显示面板的局部区域上执行。 非线性函数可以是分段线性函数或一些其它非线性函数。