公开(公告)号：US20150350641A1

公开(公告)日：2015-12-03

申请号：US14636839

申请日：2015-03-03

Applicant: Apple Inc.

Inventor： Xiaosong Zhou ,  Jiefu Zhai ,  Yeping Su ,  Chris Y. Chung ,  Hsi-Jung Wu

IPC: H04N19/105 ,  H04N19/159 ,  H04N19/13 ,  H04N19/176 ,  H04N19/18 ,  H04N19/119 ,  H04N19/167

CPC classification number: H04N19/105 ,  H04N19/119 ,  H04N19/13 ,  H04N19/159 ,  H04N19/167 ,  H04N19/176 ,  H04N19/18 ,  H04N19/85 ,  H04N19/98

Abstract: A video coding/decoding system codes data efficiently even when input video data exhibits changes in dynamic range. The system may map pixel values of the first frame from a dynamic range specific to the input image data to a second dynamic range that applies universally to a plurality of frames that have different dynamic ranges defined for them. The system may code the mapped pixel values to reduce bandwidth of the mapped frame data, and thereafter transmit the coded image data to a channel.

Abstract translation: 视频编码/解码系统即使在输入视频数据表现出动态范围的变化时，也能有效地对数据进行编 该系统可将来自特定于输入图像数据的动态范围的第一帧的像素值映射到通常适用于为它们定义的不同动态范围的多个帧的第二动态范围。 系统可以对映射的像素值进行编码以减少映射的帧数据的带宽，然后将编码的图像数据发送到信道。

公开(公告)号：US08976254B2

公开(公告)日：2015-03-10

申请号：US13691163

申请日：2012-11-30

Applicant: Apple Inc.

Inventor： Douglas Scott Price ,  Xiaosong Zhou ,  Hsi-Jung Wu

IPC: H04N5/228 ,  H04N7/01 ,  H04N19/132 ,  H04N19/136 ,  H04N19/587 ,  H04N5/77 ,  H04N19/80

CPC classification number: H04N7/0102 ,  H04N5/772 ,  H04N7/0127 ,  H04N7/0135 ,  H04N7/0137 ,  H04N19/132 ,  H04N19/136 ,  H04N19/587 ,  H04N19/80

Abstract: Embodiments of the invention provide techniques for upsampling a video sequence for coding. According to the method, an estimate of camera motion may be obtained from motion sensor data. Video data may be analyzed to detect motion within frames output from a camera that is not induced by the camera motion. When non-camera motion falls within a predetermined operational limit, video upsampling processes may be engaged. In another embodiment, video upsampling may be performed by twice estimating image content for a hypothetical new a frame using two different sources as inputs. A determination may be made whether the two estimates of the frame match each other sufficiently well. If so, the two estimates may be merged to yield a final estimated frame and the new frame may be integrated into a stream of video data.

Abstract translation: 本发明的实施例提供了用于对用于编码的视频序列进行上采样的技术。 根据该方法，可以从运动传感器数据获得相机运动的估计。 可以分析视频数据以检测来自摄像机输出的不是由摄像机运动引起的帧内的运动。 当非相机运动落在预定的操作限制内时，可以使用视频上采样处理。 在另一个实施例中，视频上采样可以通过使用两个不同的源作为输入来对假设的新帧的两次估计图像内容来执行。 可以确定帧的两个估计是否足够好地匹配。 如果是这样，则两个估计可以被合并以产生最终的估计帧，并且新的帧可以被集成到视频数据流中。

公开(公告)号：US20140362919A1

公开(公告)日：2014-12-11

申请号：US13913169

申请日：2013-06-07

Applicant: Apple Inc.

Inventor： Xiaosong Zhou ,  Hsi-Jung Wu ,  Chris Y. Chung ,  Albert E. Keinath ,  David R. Conrad ,  Yunfei Zheng ,  Dazhong Zhang ,  Jae Hoon Kim

IPC: H04N19/583

CPC classification number: H04N19/51 ,  H04N19/119 ,  H04N19/136 ,  H04N19/14 ,  H04N19/159 ,  H04N19/172 ,  H04N19/23 ,  H04N19/46

Abstract: The invention is directed to an efficient way for encoding and decoding video. Embodiments include identifying different coding units that share a similar characteristic. The characteristic can be, for example: quantization values, modes, block sizes, color space, motion vectors, depth, facial and non-facial regions, and filter values. An encoder may then group the units together as a coherence group. An encoder may similarly create a table or other data structure of the coding units. An encoder may then extract the commonly repeating characteristic or attribute from the coding units. The encoder may transmit the coherence groups along with the data structure, and other coding units which were not part of a coherence group. The decoder may receive the data, and utilize the shared characteristic by storing locally in cache, for faster repeated decoding, and decode the coherence group together.

Abstract translation: 本发明涉及用于对视频进行编码和解码的有效方式。 实施例包括识别共享相似特征的不同编码单元。 特征可以是例如：量化值，模式，块大小，颜色空间，运动矢量，深度，面部和非面部区域以及滤波器值。 然后，编码器可以将这些单元组合在一起作为一致性组。 编码器可以类似地创建编码单元的表或其他数据结构。 然后，编码器可以从编码单元中提取常见的重复特性或属性。 编码器可以连同数据结构以及不是相干组的一部分的其他编码单元一起发送相干组。 解码器可以接收数据，并通过将本地存储在高速缓存中来利用共享特性，以便更快的重复解码，并将相干组一起解码。

公开(公告)号：US20140092997A1

公开(公告)日：2014-04-03

申请号：US13632913

申请日：2012-10-01

Applicant: APPLE INC.

Inventor： Xiaosong Zhou ,  Hsi-Jung Wu

IPC: H04N7/64

CPC classification number: H04N19/65 ,  H04N19/44 ,  H04N19/67 ,  H04N19/895

Abstract: Error mitigation techniques are provided for video coding system in which input frames are selected for coding either as a Random Access Pictures (“RAP frames”) or as a non-RAP frame. Coded RAP frames may include RAP identifiers that set an ID context for subsequent frames. Coded non-RAP frames may include RAP identifiers that match the RAP identifiers that were included in the coded RAP frames. Thus, in the absence of transmission errors, a coded non-RAP frame should include a RAP identifier that matches the identifier of the preceding RAP frame. If the identifier of a non-RAP frame does not match the identifier of the RAP frame that immediately preceded it, then it indicates that a RAP frame was lost during transmission. In this case, the decoder may engage error recovery processes.

Abstract translation: 为视频编码系统提供误差减轻技术，其中输入帧被选择用于编码为随机接入图像（“RAP帧”）或作为非RAP帧。 编码的RAP帧可以包括为后续帧设置ID上下文的RAP标识符。 编码的非RAP帧可以包括与包括在经编码的RAP帧中的RAP标识符匹配的RAP标识符。 因此，在没有传输错误的情况下，编码的非RAP帧应该包括与前一个RAP帧的标识符匹配的RAP标识符。 如果非RAP帧的标识符与其之前的RAP帧的标识符不匹配，则表示在传输期间RAP帧丢失。 在这种情况下，解码器可以进行错误恢复处理。

公开(公告)号：US20130328771A1

公开(公告)日：2013-12-12

申请号：US13631263

申请日：2012-09-28

Applicant: APPLE INC.

Inventor： Yao-Chung Lin ,  Xiaosong Zhou ,  Hsi-Jung Wu

IPC: G06F3/033

CPC classification number: G06F3/03 ,  G06F1/1694 ,  G06F2200/1636 ,  H04N7/183

Abstract: Embodiments of the present invention generate estimates of device motion from two data sources on a computing device—a motion sensor and a camera. The device may compare the estimates to each other to determine if they agree. If they agree, the device may confirm that device motion estimates based on the motion sensor are accurate and may output those estimates to an application within the device. If the device motion estimates disagree, the device may alter the motion estimates obtained from the motion sensor before outputting them to the application.

Abstract translation: 本发明的实施例从计算设备上的两个数据源（运动传感器和照相机）生成设备运动的估计。 设备可以将估计值相互比较以确定它们是否一致。 如果他们同意，设备可以确认基于运动传感器的设备运动估计是准确的，并且可以将这些估计值输出到设备内的应用。 如果设备运动估计不同意，则在将其输出到应用程序之前，设备可以改变从运动传感器获得的运动估计。

公开(公告)号：US20130235928A1

公开(公告)日：2013-09-12

申请号：US13652311

申请日：2012-10-15

Applicant: APPLE, INC.

Inventor： Chris Y. Chung ,  Hao Pan ,  Jiefu Zhai ,  Yeping Su ,  Douglas Scott Price ,  Hsi-Jung Wu ,  Xiaosong Zhou

IPC: H04N7/32 ,  H04N7/26

CPC classification number: H04N19/577 ,  H04N19/107 ,  H04N19/117 ,  H04N19/124 ,  H04N19/139 ,  H04N19/149 ,  H04N19/15 ,  H04N19/154 ,  H04N19/156 ,  H04N19/172 ,  H04N19/192 ,  H04N19/46 ,  H04N19/58

Abstract: Embodiments of the present invention provide techniques for efficiently coding/decoding video data during circumstances when constraints are imposed on the video data. A frame from a video sequence may be marked as a delayed decoder refresh frame. Frames successive to the delayed decoder refresh frame in coding order may predictively coded without reference to frames preceding the delayed decoder refresh frame in coding order. The distance between the delayed decoder refresh frame and the successive frames may exceed a distance threshold. Frames successive to a current frame in decoding order may be decoded without reference to frames preceding the current frame in decoding order. The distance between the current frame and the successive frames may exceed a distance threshold.

Abstract translation: 本发明的实施例提供了在对视频数据施加约束的情况下有效地对视频数据进行编码/解码的技术。 来自视频序列的帧可被标记为延迟解码器刷新帧。 以编码顺序连续到延迟解码器刷新帧的帧可以以编码顺序预先编码而不参考延迟解码器刷新帧之前的帧。 延迟解码器刷新帧和连续帧之间的距离可以超过距离阈值。 可以解码顺序对当前帧连续的帧进行解码，而不参考当前帧之前的帧。 当前帧与连续帧之间的距离可能会超过距离阈值。

公开(公告)号：US12137199B2

公开(公告)日：2024-11-05

申请号：US18407198

申请日：2024-01-08

Applicant: Apple Inc.

Inventor： Fanyi Duanmu ,  Jun Xin ,  Hsi-Jung Wu ,  Xiaosong Zhou

IPC: H04N13/117 ,  H04N13/178 ,  H04N21/218 ,  H04N21/24

Abstract: A system obtains a data set representing immersive video content for display at a display time, including first data representing the content according to a first level of detail, and second data representing the content according to a second higher level of detail. During one or more first times prior to the display time, the system causes at least a portion of the first data to be stored in a buffer. During one or more second times prior to the display time, the system generates a prediction of a viewport for displaying the content to a user at the display time, identifies a portion of the second data corresponding to the prediction of the viewport, and causes the identified portion of the second data to be stored in the video buffer. At the display time, the system causes the content to be displayed to the user using the video buffer.

公开(公告)号：US12096044B2

公开(公告)日：2024-09-17

申请号：US18181261

申请日：2023-03-09

Applicant: Apple Inc.

Inventor： Xiaohua Yang ,  Alexandros Tourapis ,  Dazhong Zhang ,  Hang Yuan ,  Hsi-Jung Wu ,  Jae Hoon Kim ,  Jiefu Zhai ,  Ming Chen ,  Xiaosong Zhou

IPC: H04N19/90 ,  G06F3/01 ,  H04N19/52 ,  H04N19/597 ,  H04N21/2343

CPC classification number: H04N19/90 ,  G06F3/013 ,  H04N19/52 ,  H04N19/597 ,  H04N21/234345

Abstract: In communication applications, aggregate source image data at a transmitter exceeds the data that is needed to display a rendering of a viewport at a receiver. Improved streaming techniques that include estimating a location of a viewport at a future time. According to such techniques, the viewport may represent a portion of an image from a multi-directional video to be displayed at the future time, and tile(s) of the image may be identified in which the viewport is estimated to be located. In these techniques, the image data of tile(s) in which the viewport is estimated to be located may be requested at a first service tier, and the other tile in which the viewport is not estimated to be located may be requested at a second service tier, lower than the first service tier.

公开(公告)号：US20240305682A1

公开(公告)日：2024-09-12

申请号：US18600378

申请日：2024-03-08

Applicant: Apple Inc.

Inventor： Jay Mayur Khandhar ,  Borna Ghavam ,  Jinbo Qiu ,  Christopher M. Garrido ,  Karthick Santhanam ,  Patrick Miauton ,  Xiaosong Zhou ,  Dazhong Zhang ,  Kristian D. Pereira ,  Dan Miao

IPC: H04L65/75 ,  G06F3/01 ,  G06T13/40 ,  G06V10/25

CPC classification number: H04L65/762 ,  G06F3/013 ,  G06T13/40 ,  G06V10/25

Abstract: A technique for transmitting data in a copresence environment includes initiating a virtual communication session between a local device and remote devices in a shared copresence environment, where each of the plurality of sending devices are transmitting a sending quality data stream in the virtual communication session. A region of interest for the local device is determined that includes a portion of the copresence environment. The local device subscribes to a first quality data stream for the remote devices represented in the region of interest, and a second quality data stream for the remote devices not represented in the region of interest.

公开(公告)号：US20230117742A1

公开(公告)日：2023-04-20

申请号：US18083468

申请日：2022-12-16

Applicant: Apple Inc.

Inventor： Fanyi Duanmu ,  Jun Xin ,  Hsi-Jung Wu ,  Xiaosong Zhou

IPC: H04N13/117 ,  H04N21/218 ,  H04N21/24 ,  H04N13/178

Abstract: A system obtains a data set representing immersive video content for display at a display time, including first data representing the content according to a first level of detail, and second data representing the content according to a second higher level of detail. During one or more first times prior to the display time, the system causes at least a portion of the first data to be stored in a buffer. During one or more second times prior to the display time, the system generates a prediction of a viewport for displaying the content to a user at the display time, identifies a portion of the second data corresponding to the prediction of the viewport, and causes the identified portion of the second data to be stored in the video buffer. At the display time, the system causes the content to be displayed to the user using the video buffer.