公开(公告)号：US08988341B2

公开(公告)日：2015-03-24

申请号：US13631263

申请日：2012-09-28

Applicant: Apple Inc.

Inventor： Yao-Chung Lin ,  Xiaosong Zhou ,  Hsi-Jung Wu

IPC: G09G5/00 ,  G06F1/16

CPC classification number: G06F3/03 ,  G06F1/1694 ,  G06F2200/1636 ,  H04N7/183

Abstract: Embodiments of the present invention generate estimates of device motion from two data sources on a computing device—a motion sensor and a camera. The device may compare the estimates to each other to determine if they agree. If they agree, the device may confirm that device motion estimates based on the motion sensor are accurate and may output those estimates to an application within the device. If the device motion estimates disagree, the device may alter the motion estimates obtained from the motion sensor before outputting them to the application.

Abstract translation: 本发明的实施例从计算设备上的两个数据源（运动传感器和照相机）生成设备运动的估计。 设备可以将估计值相互比较以确定它们是否一致。 如果他们同意，设备可以确认基于运动传感器的设备运动估计是准确的，并且可以将这些估计值输出到设备内的应用。 如果设备运动估计不同意，则在将其输出到应用程序之前，设备可以改变从运动传感器获得的运动估计。

公开(公告)号：US08923640B1

公开(公告)日：2014-12-30

申请号：US13913169

申请日：2013-06-07

Applicant: Apple Inc.

Inventor： Xiaosong Zhou ,  Hsi-Jung Wu ,  Chris Y. Chung ,  Albert E. Keinath ,  David R. Conrad ,  Yunfei Zheng ,  Dazhong Zhang ,  Jae Hoon Kim

IPC: G06K9/62 ,  H04N19/583

CPC classification number: H04N19/51 ,  H04N19/119 ,  H04N19/136 ,  H04N19/14 ,  H04N19/159 ,  H04N19/172 ,  H04N19/23 ,  H04N19/46

Abstract: The invention is directed to an efficient way for encoding and decoding video. Embodiments include identifying different coding units that share a similar characteristic. The characteristic can be, for example: quantization values, modes, block sizes, color space, motion vectors, depth, facial and non-facial regions, and filter values. An encoder may then group the units together as a coherence group. An encoder may similarly create a table or other data structure of the coding units. An encoder may then extract the commonly repeating characteristic or attribute from the coding units. The encoder may transmit the coherence groups along with the data structure, and other coding units which were not part of a coherence group. The decoder may receive the data, and utilize the shared characteristic by storing locally in cache, for faster repeated decoding, and decode the coherence group together.

Abstract translation: 本发明涉及用于对视频进行编码和解码的有效方式。 实施例包括识别共享相似特征的不同编码单元。 特征可以是例如：量化值，模式，块大小，颜色空间，运动矢量，深度，面部和非面部区域以及滤波器值。 然后，编码器可以将这些单元组合在一起作为一致性组。 编码器可以类似地创建编码单元的表或其他数据结构。 然后，编码器可以从编码单元中提取常见的重复特性或属性。 编码器可以连同数据结构以及不是相干组的一部分的其他编码单元一起发送相干组。 解码器可以接收数据，并通过将本地存储在高速缓存中来利用共享特性，以便更快的重复解码，并将相干组一起解码。

公开(公告)号：US20140086315A1

公开(公告)日：2014-03-27

申请号：US13626730

申请日：2012-09-25

Applicant: APPLE INC.

Inventor： Dazhong Zhang ,  Xiaosong Zhou ,  Hsi-Jung Wu

IPC: H04N7/32

CPC classification number: H04N19/68 ,  H04N19/103 ,  H04N19/109 ,  H04N19/13 ,  H04N19/166 ,  H04N19/172 ,  H04N19/174 ,  H04N19/57 ,  H04N19/573 ,  H04N19/58 ,  H04N19/65 ,  H04N19/91

Abstract: Coding techniques for input video may include assigning picture identifiers to input frames in either long-form or short-form formats. If a network error has occurred that results in loss of previously-coded video data, a new input frame may be assigned a picture identifier that is coded in a long-form coding format. If no network error has occurred, the input frame may be assigned a picture identifier that is coded in a short-form coding format. Long-form coding may mitigate against loss of synchronization between an encoder and a decoder by picture identifiers.

Abstract translation: 用于输入视频的编码技术可以包括将图像标识符分配给长格式或短格式格式的输入帧。 如果发生导致先前编码的视频数据丢失的网络错误，则可以为新的输入帧分配以长格式编码格式编码的图像标识符。 如果没有发生网络错误，则可以为输入帧分配以短格式编码格式编码的图像标识符。 长格式编码可以通过图像标识符减轻编码器和解码器之间的同步丢失。

公开(公告)号：US08675740B2

公开(公告)日：2014-03-18

申请号：US13731819

申请日：2012-12-31

Applicant: Apple Inc.

Inventor： Hsi-Jung Wu ,  Barin Geoffry Haskell ,  Xiaojin Shi ,  James Oliver Normile

IPC: H04N7/12

CPC classification number: H04N19/44 ,  H04N19/149 ,  H04N19/152

Abstract: Disclosed is a system and method of controlling a video decoder, including a reviewing channel data representing coded video data generated by an encoder to identify parameters of a hypothetical reference decoder (HRD) used by the encoder during coding operations. A parameter representing an exit data rate requirement of a coded picture buffer (CPB) of the HRD is compared against exit rate performance of the video decoder. If the exit rate performance of the video coder matches the exit rate requirement of the HRD, the coded video data is decoded, otherwise, a certain decoding degradation scheme can be applied, including disabling decoder from decoding the coded video data.

Abstract translation: 公开了一种控制视频解码器的系统和方法，包括：代表由编码器产生的编码视频数据的查看频道数据，以识别由编码器在编码操作期间使用的假设参考解码器（HRD）的参数。 将表示HRD的编码图像缓冲器（CPB）的退出数据速率要求的参数与视频解码器的退出速率性能进行比较。 如果视频编码器的退出速率性能与HRD的退出率要求相匹配，则编码的视频数据被解码，否则可以应用某种解码劣化方案，包括禁止解码器解码编码的视频数据。

公开(公告)号：US20140064165A1

公开(公告)日：2014-03-06

申请号：US13667814

申请日：2012-11-02

Applicant: APPLE INC.

Inventor： Chris Y. Chung ,  Wen Zhao ,  Luciano M. Verger ,  Hyeonkuk Jeong ,  Gang Lu ,  Srinivasan Vasudevan ,  Xiaosong Zhou ,  Hsi-Jung Wu ,  James O. Normile

IPC: H04W52/02

CPC classification number: H04N7/15 ,  H04L12/1827 ,  H04L65/1069 ,  H04N5/23241 ,  H04N5/372 ,  H04W52/02 ,  H04W52/0209 ,  H04W52/0216 ,  Y02D70/00 ,  Y02D70/1224 ,  Y02D70/1242 ,  Y02D70/1262 ,  Y02D70/142 ,  Y02D70/144 ,  Y02D70/146 ,  Y02D70/164 ,  Y02D70/168 ,  Y02D70/24

Abstract: In video conferencing over a radio network, the radio equipment is a major power consumer especially in cellular networks such as LTE. In order to reduce the radio power consumption in video conferencing, it is important to introduce an enough radio inactive time. Several types of data buffering and bundling can be employed within a reasonable range of latency that doesn't significantly disrupt the real-time nature of video conferencing. In addition, the data transmission can be synchronized to the data reception in a controlled manner, which can result in an even longer radio inactive time and thus take advantage of radio power saving modes such as LTE C-DRX.

Abstract translation: 在无线电网络的视频会议中，无线电设备是主要的电力消费者，特别是在诸如LTE的蜂窝网络中。 为了降低视频会议中的无线电功耗，重要的是要引入足够的无线电时间。 几种类型的数据缓冲和捆绑可以在合理的延迟范围内使用，不会显着破坏视频会议的实时性质。 此外，可以以受控的方式将数据传输同步到数据接收，这可以导致更长的无线电时间，从而利用诸如LTE C-DRX之类的无线电节能模式。

公开(公告)号：US20130329064A1

公开(公告)日：2013-12-12

申请号：US13691163

申请日：2012-11-30

Applicant: APPLE INC.

Inventor： Douglas Scott Price ,  Xiaosong Zhou ,  Hsi-Jung Wu

IPC: H04N7/01

CPC classification number: H04N7/0102 ,  H04N5/772 ,  H04N7/0127 ,  H04N7/0135 ,  H04N7/0137 ,  H04N19/132 ,  H04N19/136 ,  H04N19/587 ,  H04N19/80

Abstract: Embodiments of the invention provide techniques for upsampling a video sequence for coding. According to the method, an estimate of camera motion may be obtained from motion sensor data. Video data may be analyzed to detect motion within frames output from a camera that is not induced by the camera motion. When non-camera motion falls within a predetermined operational limit, video upsampling processes may be engaged. In another embodiment, video upsampling may be performed by twice estimating image content for a hypothetical new a frame using two different sources as inputs. A determination may be made whether the two estimates of the frame match each other sufficiently well. If so, the two estimates may be merged to yield a final estimated frame and the new frame may be integrated into a stream of video data.

Abstract translation: 本发明的实施例提供了用于对用于编码的视频序列进行上采样的技术。 根据该方法，可以从运动传感器数据获得相机运动的估计。 可以分析视频数据以检测来自摄像机输出的不是由摄像机运动引起的帧内的运动。 当非相机运动落在预定的操作限制内时，可以使用视频上采样处理。 在另一个实施例中，视频上采样可以通过使用两个不同的源作为输入来对假设的新帧的两次估计图像内容来执行。 可以确定帧的两个估计是否足够好地匹配。 如果是这样，则两个估计可以被合并以产生最终的估计帧，并且新的帧可以被集成到视频数据流中。

公开(公告)号：US20130235931A1

公开(公告)日：2013-09-12

申请号：US13631689

申请日：2012-09-28

Applicant: APPLE INC.

Inventor： Yeping Su ,  Hsi-Jung Wu ,  Chris Y. Chung

IPC: H04N7/24 ,  H04N7/32

CPC classification number: H04N19/86 ,  H04N19/46

Abstract: A system and method is presented to mask artifacts with content-adaptive comfort noise. Encoder side analysis may determine initial comfort noise characteristics. Noise parameters may then be developed for each frame or sequence of frames that define comfort noise patches that mask the artifacts. At the decoder, a comfort noise patch can be fetched from memory or created based on the amplitude and spatial characteristics of the comfort noise specified in the noise parameters. The noise patch may additionally be scaled or otherwise adjusted to accommodate the capabilities and/or limitations of the specific decoder.

Abstract translation: 提出了一种系统和方法来掩盖具有内容自适应舒适噪声的伪影。 编码器侧分析可以确定初始舒适噪声特性。 然后，可以为定义可以掩蔽伪像的舒适噪声补丁的每个帧或帧序列开发噪声参数。 在解码器处，可以根据噪声参数中规定的舒适噪声的幅度和空间特征，从存储器中获取舒适噪声补丁或创建舒适噪声补丁。 另外可以对噪声补丁进行缩放或调整以适应特定解码器的能力和/或限制。

公开(公告)号：US20130070151A1

公开(公告)日：2013-03-21

申请号：US13677887

申请日：2012-11-15

Applicant: Apple Inc.

Inventor： Douglas S. Price ,  Xiaosong Zhou ,  Hsi-Jung Wu

IPC: H04N5/235

CPC classification number: H04N5/235 ,  H04N5/2351 ,  H04N5/243 ,  H04N2007/145

Abstract: Systems, methods, and a computer readable medium for performing auto exposure (AE) techniques that are beneficial in variable lighting conditions—and particularly applicable to handheld and/or mobile videoconferencing applications—are disclosed herein. Handheld and/or mobile videoconferencing applications—unlike their fixed camera counterparts—are often exposed to a wide variety of rapidly changing lighting and scene conditions, and thus face a difficult trade-off between adjusting exposure parameter values too frequently or not frequently enough. In personal electronic devices executing such handheld and/or mobile videoconferencing applications, it may be desirable to: use a small, centered, and center-weighted exposure metering region; set a relatively low brightness target value; and adjust the camera's exposure parameter values according to a distance-dependent convergence speed function. The use of such techniques, in conjunction with a relatively large stability region, may also improve the quality of a video encoder's temporal predictions—and thus video quality—in videoconferencing applications.

Abstract translation: 本文公开了用于执行可变照明条件有利的自动曝光（AE）技术的系统，方法和计算机可读介质，并且特别适用于手持和/或移动视频会议应用。 手持和/或移动视频会议应用程序（与固定摄像机对应的设备不同）通常会受到各种快速变化的照明和场景条件的影响，因此在过多调整曝光参数值或不够频繁地面临困难的折衷。 在执行这种手持和/或移动视频会议应用的个人电子设备中，可能需要：使用小的，中心的和中心加权的曝光计量区域; 设定相对较低的亮度目标值; 并根据距离相关的收敛速度功能调整相机的曝光参数值。 使用这样的技术结合相对较大的稳定性区域，也可以提高视频编码器在视频会议应用中的时间预测的质量，从而提高视频质量。

公开(公告)号：US12267506B2

公开(公告)日：2025-04-01

申请号：US17837846

申请日：2022-06-10

Applicant: Apple Inc.

Inventor： Mei Guo ,  Jun Xin ,  Hsi-Jung Wu ,  Alexandros Tourapis ,  Krishnakanth Rapaka

IPC: H04N19/159 ,  H04N19/176 ,  H04N19/182

Abstract: Techniques are disclosed for deriving prediction pixel blocks for use in intra-coding video and combined inter- and intra-coding video. In a first aspect, the techniques may include deriving value(s) for pixel location(s) of the prediction pixel block by, when a prediction direction vector assigned to the prediction vector points to quadrants I or III of a Cartesian plane, deriving the pixel location's value from pixel values in two regions of previously-decoded pixel data intercepted by extending the prediction direction vector in two opposite directions through the pixel location. When the prediction direction vector points toward quadrants II of the Cartesian plane, deriving the pixel location's value from pixel values in one region intercepted by the prediction direction vector through the pixel location, and from a second region intercepted by a vector that is orthogonal to the prediction direction vector.

公开(公告)号：US12137199B2

公开(公告)日：2024-11-05

申请号：US18407198

申请日：2024-01-08

Applicant: Apple Inc.

Inventor： Fanyi Duanmu ,  Jun Xin ,  Hsi-Jung Wu ,  Xiaosong Zhou

IPC: H04N13/117 ,  H04N13/178 ,  H04N21/218 ,  H04N21/24

Abstract: A system obtains a data set representing immersive video content for display at a display time, including first data representing the content according to a first level of detail, and second data representing the content according to a second higher level of detail. During one or more first times prior to the display time, the system causes at least a portion of the first data to be stored in a buffer. During one or more second times prior to the display time, the system generates a prediction of a viewport for displaying the content to a user at the display time, identifies a portion of the second data corresponding to the prediction of the viewport, and causes the identified portion of the second data to be stored in the video buffer. At the display time, the system causes the content to be displayed to the user using the video buffer.