公开(公告)号：US20230096567A1

公开(公告)日：2023-03-30

申请号：US17951919

申请日：2022-09-23

Applicant: APPLE INC.

Inventor： Alican NALCI ,  Alexandros TOURAPIS ,  Hsi-Jung WU ,  Jiefu ZHAI ,  Jingteng XUE ,  Jun XIN ,  Mei GUO ,  Xingyu ZHANG ,  Yeqing WU ,  Yunfei ZHENG ,  Jean Begaint

IPC: H04N19/147 ,  H04N19/42 ,  H04N19/172 ,  H04N19/176 ,  H04N19/186 ,  H04N19/119 ,  H04N19/91 ,  H04N19/70 ,  H04N19/124 ,  H04N19/60

Abstract: Improved neural-network-based image and video coding techniques are presented, including hybrid techniques that include both tools of a host codec and neural-network-based tools. In these improved techniques, the host coding tools may include conventional video coding standards such H.266 (VVC). In an aspects, source frames may be partitioned and either host or neural-network-based tools may be selected per partition. Coding parameter decisions for a partition may be constrained based on the partitioning and coding tool selection. Rate control for host and neural network tools may be combined. Multi-stage processing of neural network output may use a checkerboard prediction pattern.

公开(公告)号：US20200177927A1

公开(公告)日：2020-06-04

申请号：US16204792

申请日：2018-11-29

Applicant: Apple Inc.

Inventor： Xiaohua YANG ,  Alexandros TOURAPIS ,  Dazhong ZHANG ,  Hang YUAN ,  Hsi-Jung WU ,  Jae Hoon KIM ,  Jiefu ZHAI ,  Ming CHEN ,  Xiaosong ZHOU

IPC: H04N19/90 ,  H04N21/2343 ,  G06F3/01 ,  H04N19/597 ,  H04N19/52

Abstract: In communication applications, aggregate source image data at a transmitter exceeds the data that is needed to display a rendering of a viewport at a receiver. Improved streaming techniques that include estimating a location of a viewport at a future time. According to such techniques, the viewport may represent a portion of an image from a multi-directional video to be displayed at the future time, and tile(s) of the image may be identified in which the viewport is estimated to be located. In these techniques, the image data of tile(s) in which the viewport is estimated to be located may be requested at a first service tier, and the other tile in which the viewport is not estimated to be located may be requested at a second service tier, lower than the first service tier.

公开(公告)号：US20230269400A1

公开(公告)日：2023-08-24

申请号：US18181261

申请日：2023-03-09

Applicant: Apple Inc.

Inventor： Xiaohua YANG ,  Alexandros TOURAPIS ,  Dazhong ZHANG ,  Hang YUAN ,  Hsi-Jung WU ,  Jae Hoon KIM ,  Jiefu ZHAI ,  Ming CHEN ,  Xiaosong ZHOU

IPC: H04N19/90 ,  H04N19/52 ,  H04N19/597 ,  G06F3/01 ,  H04N21/2343

CPC classification number: H04N19/90 ,  H04N19/52 ,  H04N19/597 ,  G06F3/013 ,  H04N21/234345

Abstract: In communication applications, aggregate source image data at a transmitter exceeds the data that is needed to display a rendering of a viewport at a receiver. Improved streaming techniques that include estimating a location of a viewport at a future time. According to such techniques, the viewport may represent a portion of an image from a multi-directional video to be displayed at the future time, and tile(s) of the image may be identified in which the viewport is estimated to be located. In these techniques, the image data of tile(s) in which the viewport is estimated to be located may be requested at a first service tier, and the other tile in which the viewport is not estimated to be located may be requested at a second service tier, lower than the first service tier.

公开(公告)号：US20240397119A1

公开(公告)日：2024-11-28

申请号：US18797415

申请日：2024-08-07

Applicant: Apple Inc.

Inventor： Xiaohua YANG ,  Alexandros TOURAPIS ,  Dazhong ZHANG ,  Hang YUAN ,  Hsi-Jung WU ,  Jae Hoon KIM ,  Jiefu ZHAI ,  Ming CHEN ,  Xiaosong ZHOU

IPC: H04N19/90 ,  G06F3/01 ,  H04N19/52 ,  H04N19/597 ,  H04N21/2343

Abstract: In communication applications, aggregate source image data at a transmitter exceeds the data that is needed to display a rendering of a viewport at a receiver. Improved streaming techniques that include estimating a location of a viewport at a future time. According to such techniques, the viewport may represent a portion of an image from a multi-directional video to be displayed at the future time, and tile(s) of the image may be identified in which the viewport is estimated to be located. In these techniques, the image data of tile(s) in which the viewport is estimated to be located may be requested at a first service tier, and the other tile in which the viewport is not estimated to be located may be requested at a second service tier, lower than the first service tier.

公开(公告)号：US20230394081A1

公开(公告)日：2023-12-07

申请号：US18327125

申请日：2023-06-01

Applicant: Apple Inc.

Inventor： Shujie LIU ,  Xiaosong ZHOU ,  Hsi-Jung WU ,  Jiefu ZHAI ,  Ke ZHANG ,  Ming CHEN

IPC: G06F16/783 ,  G06T7/62 ,  G06F16/75

CPC classification number: G06F16/7837 ,  G06T7/62 ,  G06F16/75

Abstract: A video classification, indexing, and retrieval system is disclosed that classifies and retrieves video along multiple indexing dimensions. A search system may field queries identifying desired parameters of video, search an indexed database for videos that match the query parameters, and create clips extracted from responsive videos that are provided in response. In this manner, different queries may cause different clips to be created from a single video, each clip tailored to the parameters of the query that is received.

公开(公告)号：US20200236349A1

公开(公告)日：2020-07-23

申请号：US16254528

申请日：2019-01-22

Applicant: Apple Inc.

Inventor： Jiefu ZHAI ,  Xingyu ZHANG ,  Xiaosong ZHOU ,  Jun XIN ,  Hsi-Jung WU ,  Yeping SU

IPC: H04N19/105 ,  H04N19/147 ,  H04N19/159 ,  H04N19/176 ,  H04N19/61 ,  G06N3/08

Abstract: Systems and methods disclosed for video compression, utilizing neural networks for predictive video coding. Processes employed combine multiple banks of neural networks with codec system components to carry out the coding and decoding of video data.

公开(公告)号：US20220191473A1

公开(公告)日：2022-06-16

申请号：US17568266

申请日：2022-01-04

Applicant: Apple Inc.

Inventor： Jiefu ZHAI ,  Xingyu ZHANG ,  Xiaosong ZHOU ,  Jun XIN ,  Hsi-Jung WU ,  Yeping SU

IPC: H04N19/105 ,  H04N19/147 ,  H04N19/159 ,  H04N19/176 ,  H04N19/61 ,  G06N3/08

Abstract: Systems and methods disclosed for video compression, utilizing neural networks for predictive video coding. Processes employed combine multiple banks of neural networks with codec system components to carry out the coding and decoding of video data.

公开(公告)号：US20210397826A1

公开(公告)日：2021-12-23

申请号：US17339249

申请日：2021-06-04

Applicant: Apple Inc.

Inventor： Xiaoxia SUN ,  Jiefu ZHAI ,  Ke ZHANG ,  Xiaosong ZHOU ,  Hsi-Jung WU

IPC: G06K9/00 ,  G06K9/46 ,  G06T3/40 ,  G06N3/04

Abstract: Video object and keypoint location detection techniques are presented. The system includes a detection system for generation locations of an object's keypoints along with probabilities associated with the locations, and a stability system for stabilizing keypoint locations of the detected objects. In some aspects, the generated probabilities are two-dimensional array correspond locations within input images, and stability system fits the generated probabilities to a two-dimensional probability distribution function.

公开(公告)号：US20230396819A1

公开(公告)日：2023-12-07

申请号：US18327364

申请日：2023-06-01

Applicant: APPLE INC.

Inventor： Ke ZHANG ,  Xiaoxia SUN ,  Shujie LIU ,  Xiaosong ZHOU ,  Jian LI ,  Xun SHI ,  Jiefu ZHAI ,  Albert E KEINATH ,  Hsi-Jung WU ,  Jingteng XUE ,  Xingyu ZHANG ,  Jun XIN

IPC: H04N21/2343 ,  H04N7/01 ,  G06V40/16 ,  G06V40/20 ,  G06V30/10 ,  H04N21/231

CPC classification number: H04N21/2343 ,  H04N7/0127 ,  H04N21/231 ,  G06V40/20 ,  G06V30/10 ,  G06V40/172

Abstract: A video delivery system generates and stores reduced bandwidth videos from source video. The system may include a track generator that executes functionality of application(s) to be used at sink devices, in which the track generator generates tracks from execution of the application(s) on source video and generates tracks having a reduced data size as compared to the source video. The track generator may execute a first instance of application functionality on the source video, which identifies region(s) of interest from the source video. The track generator further may downsample the source video according to downsampling parameters, and execute a second instance of application functionality on the downsampled video. The track generator may determine, from a comparison of outputs from the first and second instances of the application, whether the output from the second instance of application functionality is within an error tolerance of the output from the first instance of application functionality. If so, the track generator may generate a track from the downsampled video. In this manner, the system generates tracks that enable reliable application operation when processed by sink devices but also have reduced size as compared to source video.

公开(公告)号：US20210185361A1

公开(公告)日：2021-06-17

申请号：US17188473

申请日：2021-03-01

Applicant: Apple Inc.

Inventor： Xiaohua YANG ,  Alexandros TOURAPIS ,  Dazhong ZHANG ,  Hang YUAN ,  Hsi-Jung WU ,  Jae Hoon KIM ,  Jiefu ZHAI ,  Ming CHEN ,  Xiaosong ZHOU

IPC: H04N19/90 ,  H04N19/52 ,  H04N19/597 ,  G06F3/01 ,  H04N21/2343

Abstract: In communication applications, aggregate source image data at a transmitter exceeds the data that is needed to display a rendering of a viewport at a receiver. Improved streaming techniques that include estimating a location of a viewport at a future time. According to such techniques, the viewport may represent a portion of an image from a multi-directional video to be displayed at the future time, and tile(s) of the image may be identified in which the viewport is estimated to be located. In these techniques, the image data of tile(s) in which the viewport is estimated to be located may be requested at a first service tier, and the other tile in which the viewport is not estimated to be located may be requested at a second service tier, lower than the first service tier.