公开(公告)号：US20230098082A1

公开(公告)日：2023-03-30

申请号：US17485298

申请日：2021-09-24

Applicant: Apple Inc.

Inventor： Sudeng Hu ,  David L. Biderman ,  Christopher M. Garrido ,  Hsi-Jung Wu ,  Xiaosong Zhou ,  Dazhong Zhang ,  Jinbo Qiu ,  Karthick Santhanam ,  Hang Yuan ,  Joshua L. Hare ,  Luciano M. Verger ,  Kevin Arthur Robertson ,  Sasanka Vemuri

IPC: H04N19/105 ,  H04N19/177 ,  H04N19/124 ,  H04N19/146 ,  H04N19/172

Abstract: In an example method, a system receives a plurality of frames of a video, and generates a data structure representing the video and representing a plurality of temporal layers. Generating the data structure includes: (i) determining a plurality of quality levels for presenting the video, where each of the quality levels corresponds to a different respective sampling period for sampling the frames of the video, (ii) assigning, based on the sampling periods, each of the frames to a respective one of the temporal layers of the data structure, and (iii) indicating, in the data structure, one or more relationships between (a) at least one the frames assigned to at least one of the temporal layers of the data structure, and (b) at least another one of the frames assigned to at least another one of the temporal layers of the data structure. Further, the system outputs the data structure.

公开(公告)号：US11606574B2

公开(公告)日：2023-03-14

申请号：US16882819

申请日：2020-05-26

Applicant: Apple Inc.

Inventor： Dazhong Zhang ,  Peikang Song ,  Beibei Wang ,  Giribalan Gopalan ,  Albert E. Keinath ,  Christopher M. Garrido ,  David R. Conrad ,  Hsi-Jung Wu ,  Ming Jin ,  Hang Yuan ,  Xiaohua Yang ,  Xiaosong Zhou ,  Vikrant Kasarabada ,  Davide Concion ,  Eric L. Chien ,  Bess C. Chan ,  Karthick Santhanam ,  Gurtej Singh Chandok

IPC: H04N19/507 ,  H04N19/65

Abstract: Techniques are disclosed for coding video data in which frames from a video source are partitioned into a plurality of tiles of common size, and the tiles are coded as a virtual video sequence according to motion-compensated prediction, each tile treated as having respective temporal location of the virtual video sequence. The coding scheme permits relative allocation of coding resources to tiles that are likely to have greater significance in a video coding session, which may lead to certain tiles that have low complexity or low motion content to be skipped during coding of the tiles for select source frames. Moreover, coding of the tiles may be ordered to achieve low coding latencies during a coding session.

公开(公告)号：US11259046B2

公开(公告)日：2022-02-22

申请号：US15433505

申请日：2017-02-15

Applicant: Apple Inc.

Inventor： Jae Hoon Kim ,  Chris Y. Chung ,  Dazhong Zhang ,  Hang Yuan ,  Hsi-Jung Wu ,  Jiefu Zhai ,  Xiaosong Zhou

IPC: H04N19/597 ,  H04N19/61 ,  H04N19/105 ,  H04N19/513 ,  H04N19/159 ,  H04N19/124 ,  H04N19/13 ,  H04N19/176 ,  H04N19/51 ,  H04N19/139 ,  H04N19/182 ,  H04N19/547 ,  H04N19/527

Abstract: Methods and Systems disclosed to counteract spatial distortions introduced by imaging processes of multi-directional video frames, where objects may be projected to spherical or equirectangular representations. Techniques provided to invert the spatial distortions in video frames used as reference picture data in predictive coding, by spatially transforming the image content of the reference picture data before this image content is being used for the prediction of input video data in prediction-based coders and decoders.

公开(公告)号：US10999602B2

公开(公告)日：2021-05-04

申请号：US15390202

申请日：2016-12-23

Applicant: Apple Inc.

Inventor： Jae Hoon Kim ,  Xiaosong Zhou ,  Dazhong Zhang ,  Hang Yuan ,  Jiefu Zhai ,  Chris Y. Chung ,  Hsi-Jung Wu

IPC: H04N19/176 ,  H04N19/105 ,  H04N19/597 ,  H04N19/547

Abstract: Techniques are disclosed for coding video data predictively based on predictions made from spherical-domain projections of input pictures to be coded and reference pictures that are prediction candidates. Spherical projection of an input picture and the candidate reference pictures may be generated. Thereafter, a search may be conducted for a match between the spherical-domain representation of a pixel block to be coded and a spherical-domain representation of the reference picture. On a match, an offset may be determined between the spherical-domain representation of the pixel block to a matching portion of the of the reference picture in the spherical-domain representation. The spherical-domain offset may be transformed to a motion vector in a source-domain representation of the input picture, and the pixel block may be coded predictively with reference to a source-domain representation of the matching portion of the reference picture.

公开(公告)号：US20200213571A1

公开(公告)日：2020-07-02

申请号：US16725245

申请日：2019-12-23

Applicant: Apple Inc.

Inventor： Jae Hoon Kim ,  Ming Chen ,  Xiaosong Zhou ,  Hsi-Jung Wu ,  Dazhong Zhang ,  Hang Yuan ,  Jiefu Zhai ,  Chris Y. Chung

IPC: H04N13/161 ,  H04N13/243 ,  H04N5/232 ,  H04N19/70 ,  H04N19/597

Abstract: Frame packing techniques are disclosed for multi-directional images and video. According to an embodiment, a multi-directional source image is reformatted into a format in which image data from opposing fields of view are represented in respective regions of the packed image as flat image content. Image data from a multi-directional field of view of the source image between the opposing fields of view are represented in another region of the packed image as equirectangular image content. It is expected that use of the formatted frame will lead to coding efficiencies when the formatted image is processed by predictive video coding techniques and the like.

公开(公告)号：US20190005709A1

公开(公告)日：2019-01-03

申请号：US15638587

申请日：2017-06-30

Applicant: Apple Inc.

Inventor： Jae Hoon Kim ,  Dazhong Zhang ,  Hang Yuan ,  Hsi-Jung Wu ,  Jiefu Zhai ,  Ming Chen ,  Xiaosong Zhou

IPC: G06T15/20 ,  H04N13/02 ,  H04N5/232 ,  H04N19/117

Abstract: Techniques are disclosed for correcting artifacts in multi-view images that include a plurality of planar views. Image content the planar views may be projected from the planar representation to a spherical projection. Thereafter, a portion of the image content may be projected from the spherical projection to a planar representation. The image content of the planar representation may be used for display. Extensions are disclosed that correct artifacts that may arise during deblocking filtering of the multi-view images.

公开(公告)号：US20190004414A1

公开(公告)日：2019-01-03

申请号：US15638848

申请日：2017-06-30

Applicant: Apple Inc.

Inventor： Jae Hoon Kim ,  Ming Chen ,  Xiaosong Zhou ,  Hsi-Jung Wu ,  Dazhong Zhang ,  Hang Yuan ,  Jiefu Zhai ,  Chris Y. Chung

IPC: G03B37/04 ,  G06T3/40 ,  H04N5/232 ,  H04N13/02 ,  H04N19/172 ,  H04N19/159

Abstract: Techniques are described for implementing format configurations for multi-directional video and for switching between them. Source images may be assigned to formats that may change during a coding session. When a change occurs between formats, video coders and decoder may transform decoded reference frames from the first format to the second format. Thereafter, new frames in the second configuration may be coded or decoded predictively using transformed reference frame(s) as source(s) of prediction. In this manner, video coders and decoders may use intra-coding techniques and achieve high efficiency in coding.

公开(公告)号：US20180262714A1

公开(公告)日：2018-09-13

申请号：US15455890

申请日：2017-03-10

Applicant: Apple Inc.

Inventor： Jae Hoon Kim ,  Chris Y. Chung ,  Dazhong Zhang ,  Hang Yuan ,  Hsi-Jung Wu ,  Xiaosong Zhou ,  Jiefu Zhai

IPC: H04N7/14 ,  H04N7/15 ,  G06T7/194 ,  G06F3/0488

CPC classification number: H04N7/147 ,  G06F3/0488 ,  H04N7/15 ,  H04N13/239 ,  H04N2007/145 ,  H04N2013/0092

Abstract: Embodiments of the present disclosure provide systems and methods for perspective shifting in a video conferencing session. In one exemplary method, a video stream may be generated. A foreground element may be identified in a frame of the video stream and distinguished from a background element of the frame. Data may be received representing a viewing condition at a terminal that will display the generated video stream. The frame of the video stream may be modified based on the received data to shift of the foreground element relative to the background element. The modified video stream may be displayed at the displaying terminal.

公开(公告)号：US20180234700A1

公开(公告)日：2018-08-16

申请号：US15433505

申请日：2017-02-15

Applicant: Apple Inc.

Inventor： Jae Hoon Kim ,  Chris Y. Chung ,  Dazhong Zhang ,  Hang Yuan ,  Hsi-Jung Wu ,  Jiefu Zhai ,  Xiaosong Zhou

IPC: H04N19/597 ,  H04N19/61 ,  H04N19/105 ,  H04N19/513 ,  H04N19/159 ,  H04N19/124 ,  H04N19/13

Abstract: Multi-directional image data often contains distortions of image content that cause problems when processed by video coders that are designed to process traditional, “flat” image content. Embodiments of the present disclosure provide techniques for coding multi-directional image data using such coders. For each pixel block in a frame to be coded, an encoder may transform reference picture data within a search window about a location of the input pixel block based on displacement respectively between the location of the input pixel block and portions of the reference picture within the search window. The encoder may perform a prediction search among the transformed reference picture data to identify a match between the input pixel block and a portion of the transformed reference picture and, when a match is identified, the encoder may code the input pixel block differentially with respect to the matching portion of the transformed reference picture. The transform may counter-act distortions imposed on image content of the reference picture data by the multi-directional format, which aligns the content with image content of the input picture. The techniques apply both for intra-coding and inter-coding.

公开(公告)号：US09992450B1

公开(公告)日：2018-06-05

申请号：US15468919

申请日：2017-03-24

Applicant: Apple Inc.

Inventor： Hang Yuan ,  Chris Y. Chung ,  Dazhong Zhang ,  Hsi-Jung Wu ,  Xiaosong Zhou ,  Jae Hoon Kim ,  Jiefu Zhai

IPC: H04N7/14 ,  H04N7/15 ,  H04N5/262 ,  G06K9/00 ,  G06K9/46

CPC classification number: G06K9/4604 ,  G06K9/00711 ,  G06K2009/4666 ,  H04N5/262 ,  H04N7/147

Abstract: Embodiments of the present disclosure provide systems and methods for background concealment in a video conferencing session. In one exemplary method, a video stream may be captured and provided to a first terminal participating in a video chat session. A background element and a foreground element may be determined in the video stream. A border region may additionally be determined in the video stream. The border region may define a boundary between the foreground element and the background element. The background region may be modified based, at least in part, on video content of the border region. The modified video stream may be transmitted to a second terminal participating in the video conferencing session.