公开(公告)号：US11677934B2

公开(公告)日：2023-06-13

申请号：US17485298

申请日：2021-09-24

Applicant: Apple Inc.

Inventor： Sudeng Hu ,  David L. Biderman ,  Christopher M. Garrido ,  Hsi-Jung Wu ,  Xiaosong Zhou ,  Dazhong Zhang ,  Jinbo Qiu ,  Karthick Santhanam ,  Hang Yuan ,  Joshua L. Hare ,  Luciano M. Verger ,  Kevin Arthur Robertson ,  Sasanka Vemuri

IPC: H04N7/12 ,  H04N19/105 ,  H04N19/177 ,  H04N19/172 ,  H04N19/146 ,  H04N19/124

CPC classification number: H04N19/105 ,  H04N19/124 ,  H04N19/146 ,  H04N19/172 ,  H04N19/177

Abstract: In an example method, a system receives a plurality of frames of a video, and generates a data structure representing the video and representing a plurality of temporal layers. Generating the data structure includes: (i) determining a plurality of quality levels for presenting the video, where each of the quality levels corresponds to a different respective sampling period for sampling the frames of the video, (ii) assigning, based on the sampling periods, each of the frames to a respective one of the temporal layers of the data structure, and (iii) indicating, in the data structure, one or more relationships between (a) at least one the frames assigned to at least one of the temporal layers of the data structure, and (b) at least another one of the frames assigned to at least another one of the temporal layers of the data structure. Further, the system outputs the data structure.

公开(公告)号：US20230098082A1

公开(公告)日：2023-03-30

申请号：US17485298

申请日：2021-09-24

Applicant: Apple Inc.

Inventor： Sudeng Hu ,  David L. Biderman ,  Christopher M. Garrido ,  Hsi-Jung Wu ,  Xiaosong Zhou ,  Dazhong Zhang ,  Jinbo Qiu ,  Karthick Santhanam ,  Hang Yuan ,  Joshua L. Hare ,  Luciano M. Verger ,  Kevin Arthur Robertson ,  Sasanka Vemuri

IPC: H04N19/105 ,  H04N19/177 ,  H04N19/124 ,  H04N19/146 ,  H04N19/172

Abstract: In an example method, a system receives a plurality of frames of a video, and generates a data structure representing the video and representing a plurality of temporal layers. Generating the data structure includes: (i) determining a plurality of quality levels for presenting the video, where each of the quality levels corresponds to a different respective sampling period for sampling the frames of the video, (ii) assigning, based on the sampling periods, each of the frames to a respective one of the temporal layers of the data structure, and (iii) indicating, in the data structure, one or more relationships between (a) at least one the frames assigned to at least one of the temporal layers of the data structure, and (b) at least another one of the frames assigned to at least another one of the temporal layers of the data structure. Further, the system outputs the data structure.

公开(公告)号：US11606574B2

公开(公告)日：2023-03-14

申请号：US16882819

申请日：2020-05-26

Applicant: Apple Inc.

Inventor： Dazhong Zhang ,  Peikang Song ,  Beibei Wang ,  Giribalan Gopalan ,  Albert E. Keinath ,  Christopher M. Garrido ,  David R. Conrad ,  Hsi-Jung Wu ,  Ming Jin ,  Hang Yuan ,  Xiaohua Yang ,  Xiaosong Zhou ,  Vikrant Kasarabada ,  Davide Concion ,  Eric L. Chien ,  Bess C. Chan ,  Karthick Santhanam ,  Gurtej Singh Chandok

IPC: H04N19/507 ,  H04N19/65

Abstract: Techniques are disclosed for coding video data in which frames from a video source are partitioned into a plurality of tiles of common size, and the tiles are coded as a virtual video sequence according to motion-compensated prediction, each tile treated as having respective temporal location of the virtual video sequence. The coding scheme permits relative allocation of coding resources to tiles that are likely to have greater significance in a video coding session, which may lead to certain tiles that have low complexity or low motion content to be skipped during coding of the tiles for select source frames. Moreover, coding of the tiles may be ordered to achieve low coding latencies during a coding session.

公开(公告)号：US11259046B2

公开(公告)日：2022-02-22

申请号：US15433505

申请日：2017-02-15

Applicant: Apple Inc.

Inventor： Jae Hoon Kim ,  Chris Y. Chung ,  Dazhong Zhang ,  Hang Yuan ,  Hsi-Jung Wu ,  Jiefu Zhai ,  Xiaosong Zhou

IPC: H04N19/597 ,  H04N19/61 ,  H04N19/105 ,  H04N19/513 ,  H04N19/159 ,  H04N19/124 ,  H04N19/13 ,  H04N19/176 ,  H04N19/51 ,  H04N19/139 ,  H04N19/182 ,  H04N19/547 ,  H04N19/527

Abstract: Methods and Systems disclosed to counteract spatial distortions introduced by imaging processes of multi-directional video frames, where objects may be projected to spherical or equirectangular representations. Techniques provided to invert the spatial distortions in video frames used as reference picture data in predictive coding, by spatially transforming the image content of the reference picture data before this image content is being used for the prediction of input video data in prediction-based coders and decoders.

公开(公告)号：US11206371B2

公开(公告)日：2021-12-21

申请号：US15495095

申请日：2017-04-24

Applicant: Apple Inc.

Inventor： Chris Y. Chung ,  Dazhong Zhang ,  Hsi-Jung Wu ,  Xiaosong Zhou

IPC: H04N7/14 ,  H04N7/15 ,  H04L29/06 ,  H04N21/242 ,  H04N21/43 ,  H04N5/14 ,  G06F40/169 ,  H04N19/136

Abstract: Techniques are disclosed for overcoming communication lag between interactive operations among devices in a streaming session. According to the techniques, a first device streaming video content to a second device and an annotation is entered to a first frame being displayed at the second device, which is communicated back to the first device. Responsive to a communication that identifies the annotation, a first device may identify an element of video content from the first frame to which the annotation applies and determine whether the identified element is present in a second frame of video content currently displayed at the first terminal. If so, the first device may display the annotation with the second frame in a location where the identified element is present. If not, the first device may display the annotation via an alternate technique.

公开(公告)号：US11165989B2

公开(公告)日：2021-11-02

申请号：US16689458

申请日：2019-11-20

Applicant: Apple Inc.

Inventor： Johnny Trenh ,  Hsi-Jung Wu ,  Sarah K. Herrlinger ,  Xiaoxia Sun ,  Ian J. Baird ,  Dazhong Zhang ,  Xiaosong Zhou ,  Christopher M. Garrido

IPC: H04N7/14 ,  G06F3/01 ,  G06N3/08 ,  H04N7/15

Abstract: Techniques are presented for managing for visual prominence of participants in a video conference, including conferences where participants communicate visually, such as with sign language. According to these techniques, a visual prominence indication of a participant in a video conference may be estimated, a video stream of the participant may be encoded, and the encoded video stream may be transmitted along with an indication of the estimated visual prominence to a receiving device in the video conference.

公开(公告)号：US10999602B2

公开(公告)日：2021-05-04

申请号：US15390202

申请日：2016-12-23

Applicant: Apple Inc.

Inventor： Jae Hoon Kim ,  Xiaosong Zhou ,  Dazhong Zhang ,  Hang Yuan ,  Jiefu Zhai ,  Chris Y. Chung ,  Hsi-Jung Wu

IPC: H04N19/176 ,  H04N19/105 ,  H04N19/597 ,  H04N19/547

Abstract: Techniques are disclosed for coding video data predictively based on predictions made from spherical-domain projections of input pictures to be coded and reference pictures that are prediction candidates. Spherical projection of an input picture and the candidate reference pictures may be generated. Thereafter, a search may be conducted for a match between the spherical-domain representation of a pixel block to be coded and a spherical-domain representation of the reference picture. On a match, an offset may be determined between the spherical-domain representation of the pixel block to a matching portion of the of the reference picture in the spherical-domain representation. The spherical-domain offset may be transformed to a motion vector in a source-domain representation of the input picture, and the pixel block may be coded predictively with reference to a source-domain representation of the matching portion of the reference picture.

公开(公告)号：US10911992B2

公开(公告)日：2021-02-02

申请号：US16145055

申请日：2018-09-27

Applicant: Apple Inc.

Inventor： Joe S. Abuan ,  Ian J. Baird ,  Xiaosong Zhou ,  Christopher M. Garrido ,  Dazhong Zhang ,  Keith W. Rauenbuehler ,  Yan Yang ,  Patrick Miauton ,  Eric L. Chien ,  Berkat S. Tung ,  Karthick Santhanam

IPC: H04W72/12 ,  H04W36/00 ,  H04L29/08 ,  H04W4/08 ,  H04L29/06 ,  H04W12/04 ,  H04W12/06 ,  G06F9/48 ,  H04W36/02 ,  H04L12/18 ,  H04L9/08 ,  H04B17/318 ,  H04B17/336

Abstract: A device implementing the subject technology may include at least one processor configured to receive a plurality of buffer reports from a plurality of other devices, each of the plurality of buffer reports indicating a current target buffer level for a respective one of the plurality of other devices. The at least one processor may be further configured to determine a group target buffer level based at least in part on the current target buffer levels of the plurality of other devices and a current target buffer level of the at least one buffer. The at least one processor may be further configured to transmit the group target buffer level to the plurality of other devices. The at least one processor may be further configured to manage an occupancy of the at least one buffer based at least in part on the group target buffer level.

公开(公告)号：US10715833B2

公开(公告)日：2020-07-14

申请号：US14289082

申请日：2014-05-28

Applicant: Apple Inc.

Inventor： Yunfei Zheng ,  Dazhong Zhang ,  Xiaosong Zhou ,  Chris Y. Chung ,  Hsi-Jung Wu

IPC: H04N19/70

Abstract: An encoding system may include a video source that captures video image, a video coder, and a controller to manage operation of the system. The video coder may encode the video image into encoded video data using a plurality of subgroup parameters corresponding to a plurality of subgroups of pixels within a group. The controller may set the subgroup parameters for at least one of the subgroups of pixels in the video coder, based upon at least one parameters corresponding to the group. A decoding system may decode the video data based upon the motion prediction parameters.

公开(公告)号：US20200213571A1

公开(公告)日：2020-07-02

申请号：US16725245

申请日：2019-12-23

Applicant: Apple Inc.

Inventor： Jae Hoon Kim ,  Ming Chen ,  Xiaosong Zhou ,  Hsi-Jung Wu ,  Dazhong Zhang ,  Hang Yuan ,  Jiefu Zhai ,  Chris Y. Chung

IPC: H04N13/161 ,  H04N13/243 ,  H04N5/232 ,  H04N19/70 ,  H04N19/597

Abstract: Frame packing techniques are disclosed for multi-directional images and video. According to an embodiment, a multi-directional source image is reformatted into a format in which image data from opposing fields of view are represented in respective regions of the packed image as flat image content. Image data from a multi-directional field of view of the source image between the opposing fields of view are represented in another region of the packed image as equirectangular image content. It is expected that use of the formatted frame will lead to coding efficiencies when the formatted image is processed by predictive video coding techniques and the like.