公开(公告)号：US20160353057A1

公开(公告)日：2016-12-01

申请号：US14727471

申请日：2015-06-01

Applicant: Apple Inc.

Inventor： Chris Y. Chung ,  Dazhong Zhang ,  Hsi-Jung Wu ,  Xiaosong Zhou

IPC: H04N7/14 ,  H04L29/06 ,  G06F17/24 ,  H04N7/15

CPC classification number: H04N7/147 ,  G06F17/241 ,  H04L65/403 ,  H04L65/601 ,  H04N5/147 ,  H04N7/155 ,  H04N21/242 ,  H04N21/4307

Abstract: Techniques are disclosed for overcoming communication lag between interactive operations among devices in a streaming session. According to the techniques, a first device streaming video content to a second device and an annotation is entered to a first frame being displayed at the second device, which is communicated back to the first device. Responsive to a communication that identifies the annotation, a first device may identify an element of video content from the first frame to which the annotation applies and determine whether the identified element is present in a second frame of video content currently displayed at the first terminal. If so, the first device may display the annotation with the second frame in a location where the identified element is present. If not, the first device may display the annotation via an alternate technique.

Abstract translation: 公开了用于克服流传输会话中的设备之间的交互操作之间的通信滞后的技术。 根据该技术，将视频内容流传输到第二设备和注释的第一设备被输入到被显示在第二设备上的第一帧，该第一帧被传送回到第一设备。 响应于识别注释的通信，第一设备可以从应用注释的第一帧识别视频内容的元素，并且确定所识别的元素是否存在于当前在第一终端显示的视频内容的第二帧中。 如果是，则第一设备可以在所存在的元素的位置中显示具有第二帧的注释。 如果不是，则第一设备可以经由替代技术来显示注释。

公开(公告)号：US09451288B2

公开(公告)日：2016-09-20

申请号：US13631793

申请日：2012-09-28

Applicant: Apple Inc.

Inventor： Hsi-Jung Wu ,  Xiaosong Zhou

IPC: H04N7/12 ,  H04N19/65 ,  H04N19/105 ,  H04N19/172 ,  H04N19/107

CPC classification number: H04N19/65 ,  H04N19/105 ,  H04N19/107 ,  H04N19/172

Abstract: A video coding system may initiate coding of a new coding session with reference to an “inferred key frame” that is known both to an encoder and a decoder before a coding session begins. The inferred key frame need not be transmitted between the encoder and decoder via the channel. Instead, the inferred key frame may be stored locally at the encoder and the decoder. Frames coded at the onset of a video coding session may be coded with reference to the inferred key frame, which increases the likelihood a decoder will receive a frame it can decode properly and accelerate the rate at which the decoder generates recovered video data. Inferred key frames may be used as prediction references to recover from transmission errors.

Abstract translation: 参考在编码会话开始之前编码器和解码器已知的“推断的关键帧”，视频编码系统可以启动对新编码会话的编码。 所推断的关键帧不需要经由信道在编码器和解码器之间传输。 相反，所推断的关键帧可以本地存储在编码器和解码器处。 可以参考推断的关键帧来对视频编码会话开始时编码的帧进行编码，这增加了解码器可以正确接收到帧的可能性，并且加速了解码器产生恢复的视频数据的速率。 可以将推断的关键帧用作从传输错误中恢复的预测参考。

公开(公告)号：US20160269733A1

公开(公告)日：2016-09-15

申请号：US15066992

申请日：2016-03-10

Applicant: Apple Inc.

Inventor： Alexandros Michael Tourapis ,  Yeping Su ,  David William Singer ,  Hsi-Jung Wu

IPC: H04N19/186 ,  H04N19/44 ,  H04N19/117

CPC classification number: H04N19/186 ,  H04N9/64 ,  H04N9/67 ,  H04N11/20 ,  H04N19/117 ,  H04N19/132 ,  H04N19/147 ,  H04N19/154 ,  H04N19/189 ,  H04N19/44 ,  H04N19/46 ,  H04N19/59 ,  H04N19/70 ,  H04N19/80 ,  H04N19/82 ,  H04N19/85

Abstract: A method of adaptive chroma downsampling is presented. The method comprises converting a source image to a converted image in an output color format, applying a plurality of downsample filters to the converted image and estimating a distortion for each filter chose the filter that produces the minimum distortion. The distortion estimation includes applying an upsample filter, and a pixel is output based on the chosen filter. Methods for closed loop conversions are also presented.

Abstract translation: 提出了一种自适应色度下采样方法。 该方法包括将源图像转换为输出颜色格式的转换图像，将多个下采样滤波器应用于转换的图像，并估计每个滤波器的失真选择产生最小失真的滤波器。 失真估计包括应用上采样滤波器，并且基于所选择的滤波器输出像素。 还提出了闭环转换的方法。

公开(公告)号：US20160073106A1

公开(公告)日：2016-03-10

申请号：US14703366

申请日：2015-05-04

Applicant: Apple Inc.

Inventor： Yeping Su ,  Hsi-Jung Wu ,  Ke Zhang ,  Chris Y. Chung ,  Xiaosong Zhou

IPC: H04N19/10

CPC classification number: H04N19/10 ,  H04N21/234363 ,  H04N21/23439 ,  H04N21/23805 ,  H04N21/8456 ,  H04N21/8543

Abstract: In a video coding system, a common video sequence is coded multiple times to yield respective instances of coded video data. Each instance may be coded according to a set coding parameters derived from a target bit rate of a respective tier of service. Each tier may be coded according to a constraint that limits a maximum coding rate of the tier to be less than a target bit rate of another predetermined tier of service. Having been coded according to the constraint facilitates dynamic switching among tiers by a requesting client device processing resources or communication bandwidth changes. Improved coding systems to switch among different coding streams may increase quality of video streamed while minimizing transmission and storage size of such content.

Abstract translation: 在视频编码系统中，公共视频序列被多次编码以产生编码视频数据的各个实例。 每个实例可以根据从相应服务等级的目标比特率导出的集合编码参数进行编码。 每个层可以根据将该层的最大编码率限制为小于另一预定服务层的目标比特率的约束进行编码。 已经根据约束进行编码有助于请求客户端设备处理资源或通信带宽变化的层之间的动态切换。 改进的在不同编码流之间切换的编码系统可以提高视频流的质量，同时最小化这些内容的传输和存储大小。

公开(公告)号：US09215655B2

公开(公告)日：2015-12-15

申请号：US14465584

申请日：2014-08-21

Applicant: Apple Inc.

Inventor： Chris Y. Chung ,  Wen Zhao ,  Luciano M. Verger ,  Hyeonkuk Jeong ,  Gang Lu ,  Srinivasan Vasudevan ,  Xiaosong Zhou ,  Hsi-Jung Wu ,  James O. Normile

IPC: H04W52/02 ,  H04N7/15

CPC classification number: H04N7/15 ,  H04L12/1827 ,  H04L65/1069 ,  H04N5/23241 ,  H04N5/372 ,  H04W52/02 ,  H04W52/0209 ,  H04W52/0216 ,  Y02D70/00 ,  Y02D70/1224 ,  Y02D70/1242 ,  Y02D70/1262 ,  Y02D70/142 ,  Y02D70/144 ,  Y02D70/146 ,  Y02D70/164 ,  Y02D70/168 ,  Y02D70/24

Abstract: In video conferencing over a radio network, the radio equipment is a major power consumer especially in cellular networks such as LTE. In order to reduce the radio power consumption in video conferencing, it is important to introduce an enough radio inactive time. Several types of data buffering and bundling can be employed within a reasonable range of latency that doesn't significantly disrupt the real-time nature of video conferencing. In addition, the data transmission can be synchronized to the data reception in a controlled manner, which can result in an even longer radio inactive time and thus take advantage of radio power saving modes such as LTE C-DRX.

Abstract translation: 在无线电网络的视频会议中，无线电设备是主要的电力消费者，特别是在诸如LTE的蜂窝网络中。 为了降低视频会议中的无线电功耗，重要的是要引入足够的无线电时间。 几种类型的数据缓冲和捆绑可以在合理的延迟范围内使用，不会显着破坏视频会议的实时性质。 此外，可以以受控的方式将数据传输同步到数据接收，这可以导致更长的无线电时间，从而利用诸如LTE C-DRX之类的无线电节能模式。

公开(公告)号：US20150350641A1

公开(公告)日：2015-12-03

申请号：US14636839

申请日：2015-03-03

Applicant: Apple Inc.

Inventor： Xiaosong Zhou ,  Jiefu Zhai ,  Yeping Su ,  Chris Y. Chung ,  Hsi-Jung Wu

IPC: H04N19/105 ,  H04N19/159 ,  H04N19/13 ,  H04N19/176 ,  H04N19/18 ,  H04N19/119 ,  H04N19/167

CPC classification number: H04N19/105 ,  H04N19/119 ,  H04N19/13 ,  H04N19/159 ,  H04N19/167 ,  H04N19/176 ,  H04N19/18 ,  H04N19/85 ,  H04N19/98

Abstract: A video coding/decoding system codes data efficiently even when input video data exhibits changes in dynamic range. The system may map pixel values of the first frame from a dynamic range specific to the input image data to a second dynamic range that applies universally to a plurality of frames that have different dynamic ranges defined for them. The system may code the mapped pixel values to reduce bandwidth of the mapped frame data, and thereafter transmit the coded image data to a channel.

Abstract translation: 视频编码/解码系统即使在输入视频数据表现出动态范围的变化时，也能有效地对数据进行编 该系统可将来自特定于输入图像数据的动态范围的第一帧的像素值映射到通常适用于为它们定义的不同动态范围的多个帧的第二动态范围。 系统可以对映射的像素值进行编码以减少映射的帧数据的带宽，然后将编码的图像数据发送到信道。

公开(公告)号：US08976254B2

公开(公告)日：2015-03-10

申请号：US13691163

申请日：2012-11-30

Applicant: Apple Inc.

Inventor： Douglas Scott Price ,  Xiaosong Zhou ,  Hsi-Jung Wu

IPC: H04N5/228 ,  H04N7/01 ,  H04N19/132 ,  H04N19/136 ,  H04N19/587 ,  H04N5/77 ,  H04N19/80

CPC classification number: H04N7/0102 ,  H04N5/772 ,  H04N7/0127 ,  H04N7/0135 ,  H04N7/0137 ,  H04N19/132 ,  H04N19/136 ,  H04N19/587 ,  H04N19/80

Abstract: Embodiments of the invention provide techniques for upsampling a video sequence for coding. According to the method, an estimate of camera motion may be obtained from motion sensor data. Video data may be analyzed to detect motion within frames output from a camera that is not induced by the camera motion. When non-camera motion falls within a predetermined operational limit, video upsampling processes may be engaged. In another embodiment, video upsampling may be performed by twice estimating image content for a hypothetical new a frame using two different sources as inputs. A determination may be made whether the two estimates of the frame match each other sufficiently well. If so, the two estimates may be merged to yield a final estimated frame and the new frame may be integrated into a stream of video data.

Abstract translation: 本发明的实施例提供了用于对用于编码的视频序列进行上采样的技术。 根据该方法，可以从运动传感器数据获得相机运动的估计。 可以分析视频数据以检测来自摄像机输出的不是由摄像机运动引起的帧内的运动。 当非相机运动落在预定的操作限制内时，可以使用视频上采样处理。 在另一个实施例中，视频上采样可以通过使用两个不同的源作为输入来对假设的新帧的两次估计图像内容来执行。 可以确定帧的两个估计是否足够好地匹配。 如果是这样，则两个估计可以被合并以产生最终的估计帧，并且新的帧可以被集成到视频数据流中。

公开(公告)号：US20150003515A1

公开(公告)日：2015-01-01

申请号：US14479014

申请日：2014-09-05

Applicant: Apple Inc.

Inventor： Jim Normile ,  Thomas Pun ,  Xiaojin Shi ,  Xin Tong ,  Hsi-Jung Wu

IPC: H04N19/154 ,  H04N19/164

CPC classification number: H04N19/154 ,  H04N19/156 ,  H04N19/164

Abstract: Techniques for encoding data based at least in part upon an awareness of the decoding complexity of the encoded data and the ability of a target decoder to decode the encoded data are disclosed. In some embodiments, a set of data is encoded based at least in part upon a state of a target decoder to which the encoded set of data is to be provided. In some embodiments, a set of data is encoded based at least in part upon the states of multiple decoders to which the encoded set of data is to be provided.

公开(公告)号：US20140362919A1

公开(公告)日：2014-12-11

申请号：US13913169

申请日：2013-06-07

Applicant: Apple Inc.

Inventor： Xiaosong Zhou ,  Hsi-Jung Wu ,  Chris Y. Chung ,  Albert E. Keinath ,  David R. Conrad ,  Yunfei Zheng ,  Dazhong Zhang ,  Jae Hoon Kim

IPC: H04N19/583

CPC classification number: H04N19/51 ,  H04N19/119 ,  H04N19/136 ,  H04N19/14 ,  H04N19/159 ,  H04N19/172 ,  H04N19/23 ,  H04N19/46

Abstract: The invention is directed to an efficient way for encoding and decoding video. Embodiments include identifying different coding units that share a similar characteristic. The characteristic can be, for example: quantization values, modes, block sizes, color space, motion vectors, depth, facial and non-facial regions, and filter values. An encoder may then group the units together as a coherence group. An encoder may similarly create a table or other data structure of the coding units. An encoder may then extract the commonly repeating characteristic or attribute from the coding units. The encoder may transmit the coherence groups along with the data structure, and other coding units which were not part of a coherence group. The decoder may receive the data, and utilize the shared characteristic by storing locally in cache, for faster repeated decoding, and decode the coherence group together.

Abstract translation: 本发明涉及用于对视频进行编码和解码的有效方式。 实施例包括识别共享相似特征的不同编码单元。 特征可以是例如：量化值，模式，块大小，颜色空间，运动矢量，深度，面部和非面部区域以及滤波器值。 然后，编码器可以将这些单元组合在一起作为一致性组。 编码器可以类似地创建编码单元的表或其他数据结构。 然后，编码器可以从编码单元中提取常见的重复特性或属性。 编码器可以连同数据结构以及不是相干组的一部分的其他编码单元一起发送相干组。 解码器可以接收数据，并通过将本地存储在高速缓存中来利用共享特性，以便更快的重复解码，并将相干组一起解码。

公开(公告)号：US20140092997A1

公开(公告)日：2014-04-03

申请号：US13632913

申请日：2012-10-01

Applicant: APPLE INC.

Inventor： Xiaosong Zhou ,  Hsi-Jung Wu

IPC: H04N7/64

CPC classification number: H04N19/65 ,  H04N19/44 ,  H04N19/67 ,  H04N19/895

Abstract: Error mitigation techniques are provided for video coding system in which input frames are selected for coding either as a Random Access Pictures (“RAP frames”) or as a non-RAP frame. Coded RAP frames may include RAP identifiers that set an ID context for subsequent frames. Coded non-RAP frames may include RAP identifiers that match the RAP identifiers that were included in the coded RAP frames. Thus, in the absence of transmission errors, a coded non-RAP frame should include a RAP identifier that matches the identifier of the preceding RAP frame. If the identifier of a non-RAP frame does not match the identifier of the RAP frame that immediately preceded it, then it indicates that a RAP frame was lost during transmission. In this case, the decoder may engage error recovery processes.

Abstract translation: 为视频编码系统提供误差减轻技术，其中输入帧被选择用于编码为随机接入图像（“RAP帧”）或作为非RAP帧。 编码的RAP帧可以包括为后续帧设置ID上下文的RAP标识符。 编码的非RAP帧可以包括与包括在经编码的RAP帧中的RAP标识符匹配的RAP标识符。 因此，在没有传输错误的情况下，编码的非RAP帧应该包括与前一个RAP帧的标识符匹配的RAP标识符。 如果非RAP帧的标识符与其之前的RAP帧的标识符不匹配，则表示在传输期间RAP帧丢失。 在这种情况下，解码器可以进行错误恢复处理。