公开(公告)号：US11627343B2

公开(公告)日：2023-04-11

申请号：US17188473

申请日：2021-03-01

Applicant: Apple Inc.

Inventor： Xiaohua Yang ,  Alexandros Tourapis ,  Dazhong Zhang ,  Hang Yuan ,  Hsi-Jung Wu ,  Jae Hoon Kim ,  Jiefu Zhai ,  Ming Chen ,  Xiaosong Zhou

IPC: H04N19/52 ,  H04N19/90 ,  H04N19/597 ,  H04N21/2343 ,  G06F3/01

Abstract: In communication applications, aggregate source image data at a transmitter exceeds the data that is needed to display a rendering of a viewport at a receiver. Improved streaming techniques that include estimating a location of a viewport at a future time. According to such techniques, the viewport may represent a portion of an image from a multi-directional video to be displayed at the future time, and tile(s) of the image may be identified in which the viewport is estimated to be located. In these techniques, the image data of tile(s) in which the viewport is estimated to be located may be requested at a first service tier, and the other tile in which the viewport is not estimated to be located may be requested at a second service tier, lower than the first service tier.

公开(公告)号：US11140407B2

公开(公告)日：2021-10-05

申请号：US16222254

申请日：2018-12-17

Applicant: Apple Inc.

Inventor： Xiang Fu ,  Xiaohua Yang ,  Linfeng Guo ,  Francesco Iacopino ,  Felix Chou ,  Ying Jian He

IPC: H04N19/172 ,  H04N19/52 ,  H04N19/176 ,  H04N19/182 ,  H04N19/55 ,  H04N19/167 ,  H04N19/119

Abstract: Techniques for encoding video with reduced frame-boundary artifacts are presented. The techniques include a video encoding method, where, when skip mode is selected as a motion prediction coding mode for a pixel block predicted from a reference block of a reference frame, estimating an amount of the reference block that extends beyond the edge of the reference frame. If the amount beyond the edge does not exceed a threshold, the prediction may be coded with skip mode. If the amount beyond the edge exceeds a threshold, the pixel block may be partitioned, and the partitioned blocks may be coded with motion prediction.

公开(公告)号：US10939139B2

公开(公告)日：2021-03-02

申请号：US16204792

申请日：2018-11-29

Applicant: Apple Inc.

Inventor： Xiaohua Yang ,  Alexandros Tourapis ,  Dazhong Zhang ,  Hang Yuan ,  Hsi-Jung Wu ,  Jae Hoon Kim ,  Jiefu Zhai ,  Ming Chen ,  Xiaosong Zhou

IPC: H04N19/52 ,  H04N19/90 ,  H04N19/597 ,  H04N21/2343 ,  G06F3/01

Abstract: In communication applications, aggregate source image data at a transmitter exceeds the data that is needed to display a rendering of a viewport at a receiver. Improved streaming techniques that include estimating a location of a viewport at a future time. According to such techniques, the viewport may represent a portion of an image from a multi-directional video to be displayed at the future time, and tile(s) of the image may be identified in which the viewport is estimated to be located. In these techniques, the image data of tile(s) in which the viewport is estimated to be located may be requested at a first service tier, and the other tile in which the viewport is not estimated to be located may be requested at a second service tier, lower than the first service tier.

公开(公告)号：US10567768B2

公开(公告)日：2020-02-18

申请号：US15487853

申请日：2017-04-14

Applicant: Apple Inc.

Inventor： Xiang Fu ,  Xiaohua Yang ,  Linfeng Guo ,  Francesco Iacopino ,  Krishna Rapaka ,  Felix Chou ,  Mukta Gore

IPC: H04N19/124 ,  H04L29/06 ,  H04N19/105 ,  H04N19/117 ,  H04N19/176 ,  H04N19/61 ,  H04N19/85

Abstract: Techniques are disclosed for developing quantization matrices for use in video coding. According to these techniques a first quantization matrix may be derived from a second quantization matrix by scaling quantization values of the second quantization matrix by scaling parameters. The scaling parameters may increase according to distance between each matrix position and a matrix origin, they may be derived from characteristics of a video sequence to be coded, or both. The first quantization matrix may be communicated to a decoder. Thereafter, a video sequence may be coded predictively. As part of the coding, pixel data of the video sequence may be transformed to a plurality of frequency domain coefficients, and the frequency domain coefficients may be quantized according to the first quantization matrix.

公开(公告)号：US20200053362A1

公开(公告)日：2020-02-13

申请号：US16100310

申请日：2018-08-10

Applicant: Apple Inc.

Inventor： Xiang Fu ,  Linfeng Guo ,  Haiyan He ,  Wei Li ,  Xu Gang Zhao ,  Hao Pan ,  Xiaohua Yang ,  Krishnakanth Rapaka ,  Munehiro Nakazato ,  Haitao Guo

IPC: H04N19/176 ,  H04N19/126 ,  G06N3/08 ,  G06T7/13

Abstract: Techniques are disclosed for coding image data adaptively at different levels of downscaling. Such techniques may involve partitioning input data into pixel blocks for coding and performing content analysis on the pixel blocks. The pixel blocks may be input to block coders that operate at different pixel block sizes, which may code the pixel blocks input to them at their respective sizes. Except when a block coder operates at the partitioning size, block coders that operate at different pixel block sizes may perform downscaling of the pixel blocks to match their size with the block coders' respective coding size. A block decoder may invert the coding operations performed by the block coders, decoding coded image data at respective pixel block sizes, then upscaling decoded image data obtained therefrom to a common pixel block size. Image reconstruction may synthesize a resultant image from the decode pixel block data output by the decoders. Quality enhancements may be added by neural network processing systems that have been trained to other segmented images.

公开(公告)号：US20190082138A1

公开(公告)日：2019-03-14

申请号：US15701388

申请日：2017-09-11

Applicant: Apple Inc.

Inventor： Hao Pan ,  Munehiro Nakazato ,  Qiang Wang ,  Vaidyanath Mani ,  Xiaohua Yang ,  Xin Wang ,  Xuemei Zhang ,  Yingjun Bai

IPC: H04N7/01 ,  G06F17/10

Abstract: Techniques for adjusting formats of images and video are presented, for example where an SDR source is presented on an HDR display, or vice versa. Techniques include deriving a conversion profile for image data where the conversion profile is responsive to parameters describing characteristics of a domain of source image data and characteristics of a domain of a target rendering environment. Some techniques include creating a tone curve from weighted basis functions.

公开(公告)号：US12096044B2

公开(公告)日：2024-09-17

申请号：US18181261

申请日：2023-03-09

Applicant: Apple Inc.

Inventor： Xiaohua Yang ,  Alexandros Tourapis ,  Dazhong Zhang ,  Hang Yuan ,  Hsi-Jung Wu ,  Jae Hoon Kim ,  Jiefu Zhai ,  Ming Chen ,  Xiaosong Zhou

IPC: H04N19/90 ,  G06F3/01 ,  H04N19/52 ,  H04N19/597 ,  H04N21/2343

CPC classification number: H04N19/90 ,  G06F3/013 ,  H04N19/52 ,  H04N19/597 ,  H04N21/234345

Abstract: In communication applications, aggregate source image data at a transmitter exceeds the data that is needed to display a rendering of a viewport at a receiver. Improved streaming techniques that include estimating a location of a viewport at a future time. According to such techniques, the viewport may represent a portion of an image from a multi-directional video to be displayed at the future time, and tile(s) of the image may be identified in which the viewport is estimated to be located. In these techniques, the image data of tile(s) in which the viewport is estimated to be located may be requested at a first service tier, and the other tile in which the viewport is not estimated to be located may be requested at a second service tier, lower than the first service tier.

公开(公告)号：US20240104766A1

公开(公告)日：2024-03-28

申请号：US18369354

申请日：2023-09-18

Applicant: Apple Inc.

Inventor： Xiaohua Yang ,  Jin Wook Chang ,  Xin Wang ,  Xuemei Zhang

IPC: G06T7/73 ,  G06F3/01

CPC classification number: G06T7/73 ,  G06F3/012 ,  G06F3/013 ,  G06T2207/20021 ,  G06T2207/30196

Abstract: In one implementation, a method is performed for generating metadata estimations based on metadata subdivisions. The method includes: obtaining an input image; obtaining metadata associated with the input image; subdividing the metadata into a plurality of metadata subdivisions; determining a viewport relative to the input image based on at least one of head pose information and eye tracking information; generating one or more metadata estimations by performing an estimation algorithm on at least a portion of the plurality of metadata subdivisions based on the viewport; and generating an output image by performing an image processing algorithm on the input image based on the one or more metadata estimations.

公开(公告)号：US20230070492A1

公开(公告)日：2023-03-09

申请号：US17886858

申请日：2022-08-12

Applicant: Apple Inc.

Inventor： Jiancong Luo ,  Dzung T. Hoang ,  Francesco Iacopino ,  Linfeng Guo ,  Mukta S. Gore ,  Ryan Baldwin ,  Supradeep T. Rangarajan ,  Xiaohua Yang

IPC: H04N19/154 ,  H04N19/124 ,  H04N19/18 ,  H04N19/176

Abstract: Techniques for reducing reduce coding artifacts in video data are disclosed. In one aspect, a frame of video data is partitioned into pixel blocks, which are organized into slices. The pixel blocks of each slice are coded by a compression algorithm and an estimate of coding artifacts in the slice is made. For slices that are estimated to possess coding artifacts, the method revises coding parameters applied to pixel blocks in the slice and recodes the slice. The method substitutes recoded slices for originally-coded slices in frame, working in a priority order from a slice with the highest estimated likelihood of coding artifacts down to slices with lower estimated likelihoods of coding artifacts, measuring changes in the frame's coding size as it goes. The likelihood of coding artifacts can be estimated from slice statistics that may be developed from a comparison of transform coefficients among the pixel blocks within a slice, from an evaluation of transform coefficients of a pixel block with a slice that is estimated to have a lowest spatial complexity, or from coded luma data of the pixel blocks within a slice. In a further aspect, slice statistics may be computed from pixel block data only for a subset of slices within a frame. Slice statistics for other slices may be derived from the statistics of neighboring slices. In another aspect, a method may revise coding parameters in iterative fashion working from an initialized value and estimate (without recoding them) data sizes of coded slices that may be obtained from the revised parameters. As the method operates, it may compare the coding parameters to parameters used in a first iteration of coding and terminate the iterative process for that slice if the first iteration coding parameters are higher than the revised parameter.

公开(公告)号：US11473971B2

公开(公告)日：2022-10-18

申请号：US16586411

申请日：2019-09-27

Applicant: Apple Inc.

Inventor： Xiaohua Yang ,  Ruiqi Hu ,  Hao Pan ,  Vaidyanath Mani ,  Qiang Wang ,  Haiyan He ,  Xin Wang ,  Gui Xie ,  Jin Wook Chang

IPC: G01J1/42 ,  G06T5/00 ,  G06T7/90 ,  G06T7/11 ,  H04N1/60

Abstract: A device comprises memory, a display characterized by a display characteristic, and processors coupled to the memory. The processors execute instructions causing the processors to receive data indicative of the display characteristic, data indicative of ambient lighting, and data indicative of content characteristics for a content item; determine a tone mapping curve for the content item based on the data indicative of content characteristics; determine a first, so-called “anchor” point along the tone mapping curve; modify a first portion of the tone mapping curve below the anchor point based on the data indicative of ambient lighting; modify a second portion of the tone mapping curve above the anchor point based on the data indicative of the display characteristic; perform tone mapping for the content item based on the modified toned mapping curve to obtain a tone mapped content item; and cause the display to display the tone mapped content item.