公开(公告)号：US11025933B2

公开(公告)日：2021-06-01

申请号：US15585581

申请日：2017-05-03

Applicant: Apple Inc.

Inventor： Dazhong Zhang ,  Hang Yuan ,  Peikang Song ,  Jae Hoon Kim ,  Xing Wen ,  Sudeng Hu ,  Xiaosong Zhou ,  Chris Y. Chung ,  Hsi-Jung Wu

IPC: H04N19/423 ,  H04N19/463 ,  H04N21/44 ,  H04N21/6379 ,  H04N21/6373

Abstract: Techniques are disclosed for managing memory allocations when coding video data according to multiple codec configurations. According to these techniques, devices may negotiate parameters of a coding session that include parameters of a plurality of different codec configurations that may be used during the coding session. A device may estimate sizes of decoded picture buffers for each of the negotiated codec configurations and allocate in its memory a portion of memory sized according to a largest size of the estimated decoded picture buffers. Thereafter, the devices may exchange coded video data. The exchange may involve decoding coded data of reference pictures and storing the decoded reference pictures in the allocated memory. During the coding session, the devices may toggle among the different negotiated codec configurations. As they do, reallocations of memory may be avoided.

公开(公告)号：US10924747B2

公开(公告)日：2021-02-16

申请号：US15443342

申请日：2017-02-27

Applicant: Apple Inc.

Inventor： Jae Hoon Kim ,  Chris Y. Chung ,  Dazhong Zhang ,  Hang Yuan ,  Hsi-Jung Wu ,  Jiefu Zhai ,  Xiaosong Zhou

IPC: H04B1/66 ,  H04N19/176 ,  H04N19/105 ,  H04N19/112 ,  H04N19/174 ,  H04N19/593 ,  H04N19/597

Abstract: Techniques are disclosed for coding and decoding video captured as cube map images. According to these techniques, padded reference images are generated for use during predicting input data. A reference image is stored in a cube map format. A padded reference image is generated from the reference image in which image data of a first view contained in reference image is replicated and placed adjacent to a second view contained in the cube map image. When coding a pixel block of an input image, a prediction search may be performed between the input pixel block and content of the padded reference image. When the prediction search identifies a match, the pixel block may be coded with respect to matching data from the padded reference image. Presence of replicated data in the padded reference image is expected to increase the likelihood that adequate prediction matches will be identified for input pixel block data, which will increase overall efficiency of the video coding.

公开(公告)号：US10812832B2

公开(公告)日：2020-10-20

申请号：US14732393

申请日：2015-06-05

Applicant: Apple Inc.

Inventor： Hang Yuan ,  Chris Y. Chung ,  Jae Hoon Kim ,  Yeping Su ,  Jiefu Zhai ,  Xiaosong Zhou ,  Hsi-Jung Wu

IPC: H04N19/85 ,  H04N19/33 ,  H04N19/88 ,  H04N19/503

Abstract: Coding techniques for image data may cause a still image to be converted to a “phantom” video sequence, which is coded by motion compensated prediction techniques. Thus, coded video data obtained from the coding operation may include temporal prediction references between frames of the video sequence. Metadata may be generated that identifies allocations of content from the still image to the frames of the video sequence. The coded data and the metadata may be transmitted to another device, whereupon it may be decoded by motion compensated prediction techniques and converted back to a still image data. Other techniques may involve coding an image in both a base layer representation and at least one coded enhancement layer representation. The enhancement layer representation may be coded predictively with reference to the base layer representation. The coded base layer representation may be partitioned into a plurality of individually-transmittable segments and stored. Prediction references of elements of the enhancement layer representation may be confined to segments of the base layer representation that correspond to a location of those elements. Meaning, when a pixel block of an enhancement layer maps to a given segment of the base layer representation, prediction references are confined to that segment and do not reference portions of the base layer representation that may be found in other segment(s).

公开(公告)号：US20200304837A1

公开(公告)日：2020-09-24

申请号：US16895815

申请日：2020-06-08

Applicant: Apple Inc.

Inventor： Yunfei Zheng ,  Dazhong Zhang ,  Xiaosong Zhou ,  Chris Y. Chung ,  Hsi-Jung Wu

IPC: H04N19/70

Abstract: An encoding system may include a video source that captures video image, a video coder, and a controller to manage operation of the system. The video coder may encode the video image into encoded video data using a plurality of subgroup parameters corresponding to a plurality of subgroups of pixels within a group. The controller may set the subgroup parameters for at least one of the subgroups of pixels in the video coder, based upon at least one parameters corresponding to the group. A decoding system may decode the video data based upon the motion prediction parameters.

公开(公告)号：US10749923B2

公开(公告)日：2020-08-18

申请号：US15169641

申请日：2016-05-31

Applicant: Apple Inc.

Inventor： Chris Y. Chung ,  Hsi-Jung Wu ,  Xiaosong Zhou ,  Jae Hoon Kim ,  Jingteng Xue

IPC: G06F15/16 ,  H04L29/06 ,  H04W88/02 ,  H04N21/2668 ,  G11B27/034 ,  H04N21/2343 ,  H04N21/258

Abstract: Methods and apparatus for contextual video content adaptation are disclosed. Video content is adapted based on any number of criteria such as a target device type, viewing conditions, network conditions or various use cases, for example. A target adaptation of content may be defined for a specified video source. For example, based on receiving a request from a portable device for a live sports feed, a shortened and reduced resolution version of the live sport feed video may be defined for the portable device. The source content may be accessed and adapted (e.g., adapted temporally, spatially, etc.) and an adapted version of content generated. For example, the source content may be cropped to a particular spatial region of interest and/or reduced in length to a particular scene. The generated adaptation may be transmitted to a device in response to the request, or stored to a storage device.

公开(公告)号：US10652578B2

公开(公告)日：2020-05-12

申请号：US15888559

申请日：2018-02-05

Applicant: Apple Inc.

Inventor： Jae Hoon Kim ,  Dazhong Zhang ,  Hang Yuan ,  Jiefu Zhai ,  Ming Chen ,  Xiaosong Zhou ,  Chris Y. Chung ,  Hsi-Jung Wu

IPC: H04N19/597 ,  H04N19/70 ,  H04N19/105 ,  H04N19/139 ,  H04N19/162 ,  H04N19/167 ,  H04N19/176 ,  H04N19/52

Abstract: Image processing techniques may accelerate coding of viewport data contained within multi-view image data. According to such techniques, an encoder may shifting content of a multi-directional image data according to the viewport location data provided by a decoder. The encoder may code the shifted multi-directional image data by predictive coding, and transmit to the decoder, the coded multi-directional image data and data identifying an amount of the shift. Doing so may move the viewport location to positions in the image data that are coded earlier than the positions that the viewport location naturally occupies and, thereby, may accelerate coding. On decode, a decoder may compare its present viewport location with viewport location data provided by the encoder with coded video data. The decoder may decode the coded video data and extract a portion of the decoded video data corresponding to a present viewport location for display.

公开(公告)号：US10575008B2

公开(公告)日：2020-02-25

申请号：US15169072

申请日：2016-05-31

Applicant: Apple Inc.

Inventor： Dazhong Zhang ,  Hsi-Jung Wu ,  Peikang Song ,  Yunfei Zheng ,  Chris Y. Chung ,  Jae Hoon Kim ,  Xiaosong Zhou

IPC: H04N19/184 ,  H04N19/44 ,  H04N19/37 ,  H04N19/40 ,  H04N19/85

Abstract: A method of managing resources on a terminal includes determining a number of downloaded video streams active at the terminal, prioritizing the active video streams, assigning a decoding quality level to each active video stream based on a priority assignment for each active video stream, and apportioning reception bandwidth to each active video stream based on an assigned quality level of each active video stream.

公开(公告)号：US10523913B2

公开(公告)日：2019-12-31

申请号：US15638709

申请日：2017-06-30

Applicant: Apple Inc.

Inventor： Jae Hoon Kim ,  Ming Chen ,  Xiaosong Zhou ,  Hsi-Jung Wu ,  Dazhong Zhang ,  Hang Yuan ,  Jiefu Zhai ,  Chris Y. Chung

IPC: H04N13/161 ,  H04N19/597 ,  H04N19/70 ,  H04N5/232 ,  H04N13/243 ,  H04N13/139

Abstract: Frame packing techniques are disclosed for multi-directional images and video. According to an embodiment, a multi-directional source image is reformatted into a format in which image data from opposing fields of view are represented in respective regions of the packed image as flat image content. Image data from a multi-directional field of view of the source image between the opposing fields of view are represented in another region of the packed image as equirectangular image content. It is expected that use of the formatted frame will lead to coding efficiencies when the formatted image is processed by predictive video coding techniques and the like.

公开(公告)号：US10466809B2

公开(公告)日：2019-11-05

申请号：US15789876

申请日：2017-10-20

Applicant: Apple Inc.

Inventor： Yao-Chung Lin ,  Xiaosong Zhou ,  Hsi-Jung Wu

IPC: G09G5/00 ,  G06F3/03 ,  G06F1/16 ,  H04N7/18

Abstract: Embodiments of the present invention generate estimates of device motion from two data sources on a computing device—a motion sensor and a camera. The device may compare the estimates to each other to determine if they agree. If they agree, the device may confirm that device motion estimates based on the motion sensor are accurate and may output those estimates to an application within the device. If the device motion estimates disagree, the device may alter the motion estimates obtained from the motion sensor before outputting them to the application.

公开(公告)号：US20190306524A1

公开(公告)日：2019-10-03

申请号：US15938158

申请日：2018-03-28

Applicant: Apple Inc.

Inventor： Xing Wen ,  Dazhong Zhang ,  Peikang Song ,  Xiaosong Zhou ,  Sudeng Hu ,  Hsi-Jung Wu ,  Jae Hoon Kim

IPC: H04N19/51 ,  H04N19/44 ,  H04N19/91 ,  H04N19/176 ,  H04N19/65

Abstract: Techniques are disclosed for coding and decoding video data using object recognition and object modeling as a basis of coding and error recovery. A video decoder may decode coded video data received from a channel. The video decoder may perform object recognition on decoded video data obtained therefrom, and, when an object is recognized in the decoded video data, the video decoder may generate a model representing the recognized object. It may store data representing the model locally. The video decoder may communicate the model data to an encoder, which may form a basis of error mitigation and recovery. The video decoder also may monitor deviation patterns in the object model and associated patterns in audio content; if/when video decoding is suspended due to operational errors, the video decoder may generate simulated video data by analyzing audio data received during the suspension period and developing video data from the data model and deviation(s) associated with patterns detected from the audio data.