公开(公告)号：US09544540B2

公开(公告)日：2017-01-10

申请号：US14296348

申请日：2014-06-04

Applicant: Apple Inc.

Inventor： Xiaosong Zhou ,  Hsi-Jung Wu ,  Chris Y. Chung ,  James O. Normile ,  Joe S. Abuan ,  Hyeonkuk Jeong ,  Yan Yang ,  Gobind Johar ,  Thomas Jansen

IPC: H04N7/14 ,  H04N7/15 ,  H04L12/58

CPC classification number: H04N7/15 ,  H04L51/04 ,  H04L51/10 ,  H04N7/147

Abstract: Computing devices may implement dynamic display of video communication data. Video communication data for a video communication may be received at a computing device where another application is currently displaying image data on an electronic display. A display location may be determined for the video communication data according to display attributes that are configured by the other application at runtime. Once determined, the video communication data may then be displayed in the determined location. In some embodiments, the video communication data may be integrated with other data displayed on the electronic display for the other application.

Abstract translation: 计算设备可以实现视频通信数据的动态显示。 用于视频通信的视频通信数据可以在其他应用当前正在电子显示器上显示图像数据的计算设备处被接收。 可以根据在运行时由其他应用配置的显示属性来确定视频通信数据的显示位置。 一旦确定，则可以在确定的位置显示视频通信数据。 在一些实施例中，视频通信数据可以与用于另一应用的电子显示器上显示的其他数据集成。

公开(公告)号：US20160358628A1

公开(公告)日：2016-12-08

申请号：US15173465

申请日：2016-06-03

Applicant: Apple Inc.

Inventor： Shujie Liu ,  Yunfei Zheng ,  Xiaosong Zhou ,  Hsi-Jung Wu ,  Ke Zhang ,  Albert E. Keinath ,  Chris Y. Chung

IPC: G11B27/031 ,  G06K9/00

CPC classification number: G11B27/031 ,  G06K9/00718 ,  G06K9/00751 ,  G06K9/00765

Abstract: Methods for organizing media data by automatically segmenting media data into hierarchical layers of scenes are described. The media data may include metadata and content having still image, video or audio data. The metadata may be content-based (e.g., differences between neighboring frames, exposure data, key frame identification data, motion data, or face detection data) or non-content-based (e.g., exposure, focus, location, time) and used to prioritize and/or classify portions of video. The metadata may be generated at the time of image capture or during post-processing. Prioritization information, such as a score for various portions of the image data may be based on the metadata and/or image data. Classification information such as the type or quality of a scene may be determined based on the metadata and/or image data. The classification and prioritization information may be metadata and may be used to organize the media data.

Abstract translation: 描述了通过将媒体数据自动分割成层级的场景来组织媒体数据的方法。 媒体数据可以包括具有静止图像，视频或音频数据的元数据和内容。 元数据可以是基于内容的（例如，相邻帧之间的差异，曝光数据，关键帧识别数据，运动数据或面部检测数据）或非基于内容的（例如，曝光，焦点，位置，时间）并且被使用 对视频的部分进行优先排序和/或分类。 元数据可以在图像捕获时或在后处理期间生成。 诸如图像数据的各个部分的得分的优先级信息可以基于元数据和/或图像数据。 可以基于元数据和/或图像数据来确定诸如场景的类型或质量的分类信息。 分类和优先化信息可以是元数据，并且可以用于组织媒体数据。

公开(公告)号：US20160357605A1

公开(公告)日：2016-12-08

申请号：US15173047

申请日：2016-06-03

Applicant: Apple Inc.

Inventor： Albert Keinath ,  Ke Zhang ,  Yunfei Zheng ,  Shujie Liu ,  Jiefu Zhai ,  Chris Y. Chung ,  Xiaosong Zhou ,  Hsi-Jung Wu

IPC: G06F9/50 ,  H04L29/08 ,  G06F9/48

CPC classification number: G06F9/5011 ,  G06F9/46 ,  G06F9/4881 ,  H04L67/34 ,  H04N21/41407 ,  H04N21/4433

Abstract: A system for processing media on a resource restricted device, the system including a memory to store data representing media assets and associated descriptors, and program instructions representing an application and a media processing system, and a processor to execute the program instructions, wherein the program instructions represent the media processing system, in response to a call from an application defining a plurality of services to be performed on an asset, determine a tiered schedule of processing operations to be performed upon the asset based on a processing budget associated therewith, and iteratively execute the processing operations on a tier-by-tier basis, unless interrupted.

Abstract translation: 一种用于在资源受限设备上处理媒体的系统，所述系统包括用于存储表示媒体资产和相关联描述符的数据的存储器，以及表示应用和媒体处理系统的程序指令以及执行所述程序指令的处理器，其中所述程序 指令代表媒体处理系统，响应于来自定义要在资产上执行的多个服务的应用的呼叫，基于与其相关联的处理预算来确定要在资产上执行的处理操作的分层调度，并且迭代地 除非中断，否则逐级执行处理操作。

公开(公告)号：US09508390B2

公开(公告)日：2016-11-29

申请号：US13941174

申请日：2013-07-12

Applicant: Apple Inc.

Inventor： David L. Biderman ,  Xiaosong Zhou ,  Douglas Scott Price ,  Yeping Su ,  Chris Y. Chung ,  Hsi-Jung Wu ,  Zhenheng Li ,  John Su ,  James D. Batson

IPC: H04N5/783 ,  G11B27/30 ,  G11B27/00 ,  H04N21/4402 ,  H04N21/6587 ,  H04N21/845

CPC classification number: H04N21/6125 ,  G06K9/00718 ,  G06K2009/00738 ,  G11B27/005 ,  G11B27/3081 ,  H04N21/262 ,  H04N21/440281 ,  H04N21/6587 ,  H04N21/8456

Abstract: System and methods for improved playback of a video stream are presented. Video snippets are identified that include a number of consecutive frames for playback. Snippets may be evenly temporally spaced in the video stream or may be content adaptive. Then the first frame of a snippet may be selected as the first frame of a scene or other appropriate stopping point. Scene detection, object detection, motion detection, video metadata, or other information generated during encoding or decoding of the video stream may aid in appropriate snippet selection.

Abstract translation: 介绍了改进视频流播放的系统和方法。 识别包括多个连续帧进行播放的视频片段。 片段可以在视频流中均匀地间隔开，或者可以是内容自适应的。 然后，片段的第一帧可以被选择为场景或其他适当停止点的第一帧。 在视频流的编码或解码过程中产生的场景检测，对象检测，运动检测，视频元数据或其他信息可能有助于适当的片段选择。

公开(公告)号：US20160182911A1

公开(公告)日：2016-06-23

申请号：US14964965

申请日：2015-12-10

Applicant: Apple Inc.

Inventor： Yeping Su ,  Chris Y. Chung ,  Hsi-Jung Wu ,  Xiaosong Zhou ,  Jiefu Zhai

IPC: H04N19/154 ,  H04N19/137 ,  H04N19/187 ,  H04N19/177 ,  H04N19/70 ,  H04N19/36 ,  H04N19/46 ,  H04N7/01 ,  H04N7/00 ,  H04N19/85 ,  H04N19/115 ,  H04N19/33

CPC classification number: H04N19/154 ,  H04N7/0127 ,  H04N19/115 ,  H04N19/117 ,  H04N19/137 ,  H04N19/177 ,  H04N19/187 ,  H04N19/31 ,  H04N19/46 ,  H04N19/587 ,  H04N19/85

Abstract: Judder artifacts are remedied in video coding system by employing frame rate conversion at an encoder. According to the disclosure, a source video sequence may be coded as base layer coded video at a first frame rate. An encoder may identify a portion of the coded video sequence that likely will exhibit judder effects when decoded. For those portions that likely will exhibit judder effects, video data representing the portion of the source video may be coded at a higher frame rate than a frame rate of the coded base layer data as enhancement layer data. Moreover, an encoder may generate metadata representing “FRC hints”—techniques that a decoder should employ when performing decoder-side frame rate conversion. An encoding terminal may transmit the base layer coded video and either the enhancement layer coded video or the FRC hints to a decoder. Thus, encoder infrastructure may mitigate against judder artifacts that may arise during decoding.

Abstract translation: 通过在编码器处采用帧速率转换，在视频编码系统中纠正了歪曲伪像。 根据本公开，源视频序列可以以第一帧速率被编码为基本层编码的视频。 编码器可以识别编码视频序列的一部分，其在解码时可能会呈现抖动效应。 对于可能出现抖动效应的那些部分，可以以比作为增强层数据的编码的基本层数据的帧速率更高的帧速率对表示源视频的部分的视频数据进行编码。 此外，编码器可以产生表示“FRC提示”的元数据，技术在解码器侧帧速率转换时解码器应采用的技术。 编码终端可以将基础层编码视频和增强层编码视频或FRC提示发送到解码器。 因此，编码器基础设施可以减轻在解码期间可能出现的抖动伪影。

公开(公告)号：US20160173817A1

公开(公告)日：2016-06-16

申请号：US15043448

申请日：2016-02-12

Applicant: Apple Inc.

Inventor： Jiefu Zhai ,  Chris Y. Chung ,  Xiaosong Zhou ,  Hsi-Jung Wu

IPC: H04N7/14 ,  H04N19/117

CPC classification number: H04N7/141 ,  H04N19/117

Abstract: A system may include a receiver, a decoder, a post-processor, and a controller. The receiver may receive encoded video data. The decoder may decode the encoded video data. The post-processor may perform post-processing on frames of decoded video sequence from the decoder. The controller may adjust post-processing of a current frame, based upon at least one condition parameters detected at the system.

Abstract translation: 系统可以包括接收器，解码器，后处理器和控制器。 接收机可以接收编码的视频数据。 解码器可以解码编码的视频数据。 后处理器可以对来自解码器的解码视频序列的帧执行后处理。 控制器可以基于在系统处检测到的至少一个条件参数来调整当前帧的后处理。

公开(公告)号：US20160127738A1

公开(公告)日：2016-05-05

申请号：US14704707

申请日：2015-05-05

Applicant: Apple Inc.

Inventor： Yeping Su ,  Chris Y. Chung ,  Hsi-Jung Wu ,  Jiefu Zhai ,  Ke Zhang ,  Xiaosong Zhou

IPC: H04N19/503 ,  H04N19/172 ,  H04N19/124

CPC classification number: H04N19/503 ,  H04N19/124 ,  H04N19/147 ,  H04N19/172 ,  H04N19/30

Abstract: A scalable coding system codes video as a base layer representation and an enhancement layer representation. A base layer coder may code an LDR representation of a source video. A predictor may predict an HDR representation of the source video from the coded base layer data. A comparator may generate prediction residuals which represent a difference between an HDR representation of the source video and the predicted HDR representation of the source video. A quantizer may quantize the residuals down to an LDR representation. An enhancement layer coder may code the LDR residuals. In other embodiments, the enhancement layer coder may code LDR-converted HDR video directly.

Abstract translation: 可扩展编码系统将视频编码为基本层表示和增强层表示。 基本层编码器可以对源视频的LDR表示进行编码。 预测器可以从编码的基础层数据预测源视频的HDR表示。 比较器可以产生表示源视频的HDR表示与源视频的预测HDR表示之间的差异的预测残差。 量化器可以将残差量化到LDR表示。 增强层编码器可以对LDR残差进行编码。 在其他实施例中，增强层编码器可以直接编码LDR转换的HDR视频。

公开(公告)号：US20160100131A1

公开(公告)日：2016-04-07

申请号：US14968025

申请日：2015-12-14

Applicant: Apple Inc.

Inventor： Chris Y. Chung ,  Wen Zhao ,  Luciano M. Verger ,  Hyeonkuk Jeong ,  Gang Lu ,  Srinivasan Vasudevan ,  Xiaosong Zhou ,  Hsi-Jung Wu ,  James O. Normile

IPC: H04N7/15 ,  H04N5/232 ,  H04N5/372 ,  H04L29/06

CPC classification number: H04N7/15 ,  H04L12/1827 ,  H04L65/1069 ,  H04N5/23241 ,  H04N5/372 ,  H04W52/02 ,  H04W52/0209 ,  H04W52/0216 ,  Y02D70/00 ,  Y02D70/1224 ,  Y02D70/1242 ,  Y02D70/1262 ,  Y02D70/142 ,  Y02D70/144 ,  Y02D70/146 ,  Y02D70/164 ,  Y02D70/168 ,  Y02D70/24

Abstract: In video conferencing over a radio network, the radio equipment is a major power consumer especially in cellular networks such as LTE. In order to reduce the radio power consumption in video conferencing, it is important to introduce an enough radio inactive time. Several types of data buffering and bundling can be employed within a reasonable range of latency that doesn't significantly disrupt the real-time nature of video conferencing. In addition, the data transmission can be synchronized to the data reception in a controlled manner, which can result in an even longer radio inactive time and thus take advantage of radio power saving modes such as LTE C-DRX.

Abstract translation: 在无线电网络的视频会议中，无线电设备是主要的电力消费者，尤其是在诸如LTE的蜂窝网络中。 为了降低视频会议中的无线电功耗，重要的是要引入足够的无线电时间。 几种类型的数据缓冲和捆绑可以在合理的延迟范围内使用，不会显着破坏视频会议的实时性质。 此外，可以以受控的方式将数据传输同步到数据接收，这可以导致更长的无线电时间，从而利用诸如LTE C-DRX之类的无线电节能模式。

公开(公告)号：US20150358582A1

公开(公告)日：2015-12-10

申请号：US14296354

申请日：2014-06-04

Applicant: Apple Inc.

Inventor： Xiaosong Zhou ,  Hsi-Jung Wu ,  Chris Y. Chung ,  James O. Normile ,  Joe S. Abuan ,  Hyeonkuk Jeong ,  Yan Yang ,  Gobind Johar ,  Thomas Jansen

IPC: H04N7/15

CPC classification number: H04N7/15 ,  H04L12/1818 ,  H04L51/04 ,  H04L51/046 ,  H04L51/10 ,  H04L65/1089 ,  H04L65/403 ,  H04N7/147

Abstract: Computing devices may implement dynamic transitions from video messages to video communications. Video communication data for a video message may be received at a recipient device. The video communication data may be displayed as it is received, and recorded for subsequent playback. An indication of a selection to establish a video communication with the sender of the video message may be received, or an indication that display of the video communication is to be ceased may be received. If a video communication is to be established, then a video communication connection with the sender of the video message may be created so that subsequent video communication data may be sent via the established connection.

Abstract translation: 计算设备可以实现从视频消息到视频通信的动态转换。 用于视频消息的视频通信数据可以在接收设备处被接收。 视频通信数据可以在它被接收时被显示，并被记录以用于随后的重放。 可以接收到与视频消息的发送者建立视频通信的选择的指示，或者可以接收到要停止视频通信的显示的指示。 如果要建立视频通信，则可以创建与视频消息的发送者的视频通信连接，以便可以经由建立的连接发送后续的视频通信数据。

公开(公告)号：US20150350646A1

公开(公告)日：2015-12-03

申请号：US14289082

申请日：2014-05-28

Applicant: Apple Inc.

Inventor： Yunfei Zheng ,  Dazhong Zhang ,  Xiaosong Zhou ,  Chris Y. Chung ,  Hsi-Jung Wu

IPC: H04N19/115 ,  H04N19/44 ,  H04N19/176 ,  H04N19/96 ,  H04N19/157

CPC classification number: H04N19/70

Abstract: An encoding system may include a video source that captures video image, a video coder, and a controller to manage operation of the system. The video coder may encode the video image into encoded video data using a plurality of subgroup parameters corresponding to a plurality of subgroups of pixels within a group. The controller may set the subgroup parameters for at least one of the subgroups of pixels in the video coder, based upon at least one parameters corresponding to the group. A decoding system may decode the video data based upon the motion prediction parameters.

Abstract translation: 编码系统可以包括捕获视频图像的视频源，视频编码器和用于管理系统的操作的控制器。 视频编码器可以使用与组内的多个像素组对应的多个子组参数将视频图像编码成编码视频数据。 基于对应于该组的至少一个参数，控制器可以为视频编码器中的至少一个像素组设置子组参数。 解码系统可以基于运动预测参数解码视频数据。