公开(公告)号：US08923640B1

公开(公告)日：2014-12-30

申请号：US13913169

申请日：2013-06-07

Applicant: Apple Inc.

Inventor： Xiaosong Zhou ,  Hsi-Jung Wu ,  Chris Y. Chung ,  Albert E. Keinath ,  David R. Conrad ,  Yunfei Zheng ,  Dazhong Zhang ,  Jae Hoon Kim

IPC: G06K9/62 ,  H04N19/583

CPC classification number: H04N19/51 ,  H04N19/119 ,  H04N19/136 ,  H04N19/14 ,  H04N19/159 ,  H04N19/172 ,  H04N19/23 ,  H04N19/46

Abstract: The invention is directed to an efficient way for encoding and decoding video. Embodiments include identifying different coding units that share a similar characteristic. The characteristic can be, for example: quantization values, modes, block sizes, color space, motion vectors, depth, facial and non-facial regions, and filter values. An encoder may then group the units together as a coherence group. An encoder may similarly create a table or other data structure of the coding units. An encoder may then extract the commonly repeating characteristic or attribute from the coding units. The encoder may transmit the coherence groups along with the data structure, and other coding units which were not part of a coherence group. The decoder may receive the data, and utilize the shared characteristic by storing locally in cache, for faster repeated decoding, and decode the coherence group together.

Abstract translation: 本发明涉及用于对视频进行编码和解码的有效方式。 实施例包括识别共享相似特征的不同编码单元。 特征可以是例如：量化值，模式，块大小，颜色空间，运动矢量，深度，面部和非面部区域以及滤波器值。 然后，编码器可以将这些单元组合在一起作为一致性组。 编码器可以类似地创建编码单元的表或其他数据结构。 然后，编码器可以从编码单元中提取常见的重复特性或属性。 编码器可以连同数据结构以及不是相干组的一部分的其他编码单元一起发送相干组。 解码器可以接收数据，并通过将本地存储在高速缓存中来利用共享特性，以便更快的重复解码，并将相干组一起解码。

公开(公告)号：US20140086315A1

公开(公告)日：2014-03-27

申请号：US13626730

申请日：2012-09-25

Applicant: APPLE INC.

Inventor： Dazhong Zhang ,  Xiaosong Zhou ,  Hsi-Jung Wu

IPC: H04N7/32

CPC classification number: H04N19/68 ,  H04N19/103 ,  H04N19/109 ,  H04N19/13 ,  H04N19/166 ,  H04N19/172 ,  H04N19/174 ,  H04N19/57 ,  H04N19/573 ,  H04N19/58 ,  H04N19/65 ,  H04N19/91

Abstract: Coding techniques for input video may include assigning picture identifiers to input frames in either long-form or short-form formats. If a network error has occurred that results in loss of previously-coded video data, a new input frame may be assigned a picture identifier that is coded in a long-form coding format. If no network error has occurred, the input frame may be assigned a picture identifier that is coded in a short-form coding format. Long-form coding may mitigate against loss of synchronization between an encoder and a decoder by picture identifiers.

Abstract translation: 用于输入视频的编码技术可以包括将图像标识符分配给长格式或短格式格式的输入帧。 如果发生导致先前编码的视频数据丢失的网络错误，则可以为新的输入帧分配以长格式编码格式编码的图像标识符。 如果没有发生网络错误，则可以为输入帧分配以短格式编码格式编码的图像标识符。 长格式编码可以通过图像标识符减轻编码器和解码器之间的同步丢失。

公开(公告)号：US20140064165A1

公开(公告)日：2014-03-06

申请号：US13667814

申请日：2012-11-02

Applicant: APPLE INC.

Inventor： Chris Y. Chung ,  Wen Zhao ,  Luciano M. Verger ,  Hyeonkuk Jeong ,  Gang Lu ,  Srinivasan Vasudevan ,  Xiaosong Zhou ,  Hsi-Jung Wu ,  James O. Normile

IPC: H04W52/02

CPC classification number: H04N7/15 ,  H04L12/1827 ,  H04L65/1069 ,  H04N5/23241 ,  H04N5/372 ,  H04W52/02 ,  H04W52/0209 ,  H04W52/0216 ,  Y02D70/00 ,  Y02D70/1224 ,  Y02D70/1242 ,  Y02D70/1262 ,  Y02D70/142 ,  Y02D70/144 ,  Y02D70/146 ,  Y02D70/164 ,  Y02D70/168 ,  Y02D70/24

Abstract: In video conferencing over a radio network, the radio equipment is a major power consumer especially in cellular networks such as LTE. In order to reduce the radio power consumption in video conferencing, it is important to introduce an enough radio inactive time. Several types of data buffering and bundling can be employed within a reasonable range of latency that doesn't significantly disrupt the real-time nature of video conferencing. In addition, the data transmission can be synchronized to the data reception in a controlled manner, which can result in an even longer radio inactive time and thus take advantage of radio power saving modes such as LTE C-DRX.

Abstract translation: 在无线电网络的视频会议中，无线电设备是主要的电力消费者，特别是在诸如LTE的蜂窝网络中。 为了降低视频会议中的无线电功耗，重要的是要引入足够的无线电时间。 几种类型的数据缓冲和捆绑可以在合理的延迟范围内使用，不会显着破坏视频会议的实时性质。 此外，可以以受控的方式将数据传输同步到数据接收，这可以导致更长的无线电时间，从而利用诸如LTE C-DRX之类的无线电节能模式。

公开(公告)号：US20130329064A1

公开(公告)日：2013-12-12

申请号：US13691163

申请日：2012-11-30

Applicant: APPLE INC.

Inventor： Douglas Scott Price ,  Xiaosong Zhou ,  Hsi-Jung Wu

IPC: H04N7/01

CPC classification number: H04N7/0102 ,  H04N5/772 ,  H04N7/0127 ,  H04N7/0135 ,  H04N7/0137 ,  H04N19/132 ,  H04N19/136 ,  H04N19/587 ,  H04N19/80

Abstract: Embodiments of the invention provide techniques for upsampling a video sequence for coding. According to the method, an estimate of camera motion may be obtained from motion sensor data. Video data may be analyzed to detect motion within frames output from a camera that is not induced by the camera motion. When non-camera motion falls within a predetermined operational limit, video upsampling processes may be engaged. In another embodiment, video upsampling may be performed by twice estimating image content for a hypothetical new a frame using two different sources as inputs. A determination may be made whether the two estimates of the frame match each other sufficiently well. If so, the two estimates may be merged to yield a final estimated frame and the new frame may be integrated into a stream of video data.

Abstract translation: 本发明的实施例提供了用于对用于编码的视频序列进行上采样的技术。 根据该方法，可以从运动传感器数据获得相机运动的估计。 可以分析视频数据以检测来自摄像机输出的不是由摄像机运动引起的帧内的运动。 当非相机运动落在预定的操作限制内时，可以使用视频上采样处理。 在另一个实施例中，视频上采样可以通过使用两个不同的源作为输入来对假设的新帧的两次估计图像内容来执行。 可以确定帧的两个估计是否足够好地匹配。 如果是这样，则两个估计可以被合并以产生最终的估计帧，并且新的帧可以被集成到视频数据流中。

公开(公告)号：US20130070151A1

公开(公告)日：2013-03-21

申请号：US13677887

申请日：2012-11-15

Applicant: Apple Inc.

Inventor： Douglas S. Price ,  Xiaosong Zhou ,  Hsi-Jung Wu

IPC: H04N5/235

CPC classification number: H04N5/235 ,  H04N5/2351 ,  H04N5/243 ,  H04N2007/145

Abstract: Systems, methods, and a computer readable medium for performing auto exposure (AE) techniques that are beneficial in variable lighting conditions—and particularly applicable to handheld and/or mobile videoconferencing applications—are disclosed herein. Handheld and/or mobile videoconferencing applications—unlike their fixed camera counterparts—are often exposed to a wide variety of rapidly changing lighting and scene conditions, and thus face a difficult trade-off between adjusting exposure parameter values too frequently or not frequently enough. In personal electronic devices executing such handheld and/or mobile videoconferencing applications, it may be desirable to: use a small, centered, and center-weighted exposure metering region; set a relatively low brightness target value; and adjust the camera's exposure parameter values according to a distance-dependent convergence speed function. The use of such techniques, in conjunction with a relatively large stability region, may also improve the quality of a video encoder's temporal predictions—and thus video quality—in videoconferencing applications.

Abstract translation: 本文公开了用于执行可变照明条件有利的自动曝光（AE）技术的系统，方法和计算机可读介质，并且特别适用于手持和/或移动视频会议应用。 手持和/或移动视频会议应用程序（与固定摄像机对应的设备不同）通常会受到各种快速变化的照明和场景条件的影响，因此在过多调整曝光参数值或不够频繁地面临困难的折衷。 在执行这种手持和/或移动视频会议应用的个人电子设备中，可能需要：使用小的，中心的和中心加权的曝光计量区域; 设定相对较低的亮度目标值; 并根据距离相关的收敛速度功能调整相机的曝光参数值。 使用这样的技术结合相对较大的稳定性区域，也可以提高视频编码器在视频会议应用中的时间预测的质量，从而提高视频质量。

公开(公告)号：US12192440B2

公开(公告)日：2025-01-07

申请号：US17568266

申请日：2022-01-04

Applicant: Apple Inc.

Inventor： Jiefu Zhai ,  Xingyu Zhang ,  Xiaosong Zhou ,  Jun Xin ,  Hsi-Jung Wu ,  Yeping Su

IPC: H04N19/105 ,  G06N3/08 ,  H04N19/147 ,  H04N19/159 ,  H04N19/176 ,  H04N19/61

Abstract: Systems and methods disclosed for video compression, utilizing neural networks for predictive video coding. Processes employed combine multiple banks of neural networks with codec system components to carry out the coding and decoding of video data.

公开(公告)号：US11570417B2

公开(公告)日：2023-01-31

申请号：US17326204

申请日：2021-05-20

Applicant: Apple Inc.

Inventor： Fanyi Duanmu ,  Jun Xin ,  Hsi-Jung Wu ,  Xiaosong Zhou

IPC: H04N13/00 ,  H04N21/24 ,  H04N21/218 ,  H04N13/117 ,  H04N13/178

Abstract: A system obtains a data set representing immersive video content for display at a display time, including first data representing the content according to a first level of detail, and second data representing the content according to a second higher level of detail. During one or more first times prior to the display time, the system causes at least a portion of the first data to be stored in a buffer. During one or more second times prior to the display time, the system generates a prediction of a viewport for displaying the content to a user at the display time, identifies a portion of the second data corresponding to the prediction of the viewport, and causes the identified portion of the second data to be stored in the video buffer. At the display time, the system causes the content to be displayed to the user using the video buffer.

公开(公告)号：US20220286644A1

公开(公告)日：2022-09-08

申请号：US17826116

申请日：2022-05-26

Applicant: Apple Inc.

Inventor： Xiaosong Zhou ,  Hsi-Jung Wu ,  Chris Y. Chung ,  James Normile ,  Joe S. Abuan ,  Hyeonkuk Jeong ,  Yan Yang ,  Gobind Johar ,  Thomas Christopher Jansen

IPC: H04N7/14 ,  H04L65/1069 ,  H04N7/15 ,  H04L51/10 ,  H04L51/04

Abstract: Computing devices may implement instant video communication connections for video communications. Connection information for mobile computing devices may be maintained. A request to initiate an instant video communication may be received, and if authorized, the connection information for the particular recipient mobile computing device may be accessed. Video communication data may then be sent to the recipient mobile computing device according to the connection information so that the video communication data may be displayed at the recipient device as it is received. New connection information for different mobile computing devices may be added, or updates to existing connection information may also be performed. Connection information for some mobile computing devices may be removed.

公开(公告)号：US11282543B2

公开(公告)日：2022-03-22

申请号：US15917441

申请日：2018-03-09

Applicant: Apple Inc.

Inventor： Hang Yuan ,  Jiefu Zhai ,  Ming Chen ,  Jae Hoon Kim ,  Dazhong Zhang ,  Xiaosong Zhou ,  Chris Y. Chung ,  Hsi-Jung Wu

IPC: G11B27/031 ,  G06T5/00 ,  G06T7/73 ,  G06K9/00 ,  G06T19/20

Abstract: Techniques are presented for modifying images of an object in video, for example to correct for lens distortion, or to beautify a face. These techniques include extracting and validating features of an object from a source video frame, tracking those features over time, estimating a pose of the object, modifying a 3D model of the object based on the features, and rendering a modified video frame based on the modified 3D model and modified intrinsic and extrinsic matrices. These techniques may be applied in real-time to an object in a sequence of video frames.

公开(公告)号：US20210321133A1

公开(公告)日：2021-10-14

申请号：US17207267

申请日：2021-03-19

Applicant: Apple Inc.

Inventor： Jae Hoon Kim ,  Xiaosong Zhou ,  Dazhong Zhang ,  Hang Yuan ,  Jiefu Zhai ,  Chris Y. Chung ,  Hsi-Jung Wu

IPC: H04N19/597 ,  H04N19/176 ,  H04N19/105 ,  H04N19/547

Abstract: Techniques are disclosed for coding video data predictively based on predictions made from spherical-domain projections of input pictures to be coded and reference pictures that are prediction candidates. Spherical projection of an input picture and the candidate reference pictures may be generated. Thereafter, a search may be conducted for a match between the spherical-domain representation of a pixel block to be coded and a spherical-domain representation of the reference picture. On a match, an offset may be determined between the spherical-domain representation of the pixel block to a matching portion of the of the reference picture in the spherical-domain representation. The spherical-domain offset may be transformed to a motion vector in a source-domain representation of the input picture, and the pixel block may be coded predictively with reference to a source-domain representation of the matching portion of the reference picture.