公开(公告)号：US20240062413A1

公开(公告)日：2024-02-22

申请号：US18384352

申请日：2023-10-26

Applicant: Apple Inc.

Inventor： Jeffrey S. Norris ,  Alexandre Da Veiga ,  Bruno M. Sommer ,  Ye Cong ,  Tobias Eble ,  Moinul Khan ,  Nicolas Bonnier ,  Hao Pan

IPC: G06T7/73 ,  G02B27/01 ,  G06V20/20 ,  G06V10/46 ,  G06V10/24

CPC classification number: G06T7/73 ,  G02B27/017 ,  G06V20/20 ,  G06V10/462 ,  G06V10/245 ,  G06T2207/30204

Abstract: A method includes obtaining first pass-through image data characterized by a first pose. The method includes obtaining respective pixel characterization vectors for pixels in the first pass-through image data. The method includes identifying a feature of an object within the first pass-through image data in accordance with a determination that pixel characterization vectors for the feature satisfy a feature confidence threshold. The method includes displaying the first pass-through image data and an AR display marker that corresponds to the feature. The method includes obtaining second pass-through image data characterized by a second pose. The method includes transforming the AR display marker to a position associated with the second pose in order to track the feature. The method includes displaying the second pass-through image data and maintaining display of the AR display marker that corresponds to the feature of the object based on the transformation.

公开(公告)号：US20230351672A1

公开(公告)日：2023-11-02

申请号：US18344294

申请日：2023-06-29

Applicant: Apple Inc.

Inventor： Arthur Y. Zhang ,  Ray L. Chang ,  Timothy R. Oriol ,  Ling Su ,  Gurjeet S. Saund ,  Guy Cote ,  Jim C. Chou ,  Hao Pan ,  Tobias Eble ,  Avi Bar-Zeev ,  Sheng Zhang ,  Justin A. Hensley ,  Geoffrey Stahl

IPC: H04W76/10 ,  G06T9/00 ,  G06T15/00 ,  G06F3/01 ,  G06T3/00

CPC classification number: G06T15/005 ,  G06F3/012 ,  G06T3/0093 ,  G06T9/00 ,  H04W76/10 ,  H04W88/08

Abstract: A mixed reality system that includes a device and a base station that communicate via a wireless connection The device may include sensors that collect information about the user’s environment and about the user. The information collected by the sensors may be transmitted to the base station via the wireless connection. The base station renders frames or slices based at least in part on the sensor information received from the device, encodes the frames or slices, and transmits the compressed frames or slices to the device for decoding and display. The base station may provide more computing power than conventional stand-alone systems, and the wireless connection does not tether the device to the base station as in conventional tethered systems. The system may implement methods and apparatus to maintain a target frame rate through the wireless link and to minimize latency in frame rendering, transmittal, and display.

公开(公告)号：US11473971B2

公开(公告)日：2022-10-18

申请号：US16586411

申请日：2019-09-27

Applicant: Apple Inc.

Inventor： Xiaohua Yang ,  Ruiqi Hu ,  Hao Pan ,  Vaidyanath Mani ,  Qiang Wang ,  Haiyan He ,  Xin Wang ,  Gui Xie ,  Jin Wook Chang

IPC: G01J1/42 ,  G06T5/00 ,  G06T7/90 ,  G06T7/11 ,  H04N1/60

Abstract: A device comprises memory, a display characterized by a display characteristic, and processors coupled to the memory. The processors execute instructions causing the processors to receive data indicative of the display characteristic, data indicative of ambient lighting, and data indicative of content characteristics for a content item; determine a tone mapping curve for the content item based on the data indicative of content characteristics; determine a first, so-called “anchor” point along the tone mapping curve; modify a first portion of the tone mapping curve below the anchor point based on the data indicative of ambient lighting; modify a second portion of the tone mapping curve above the anchor point based on the data indicative of the display characteristic; perform tone mapping for the content item based on the modified toned mapping curve to obtain a tone mapped content item; and cause the display to display the tone mapped content item.

公开(公告)号：US11295425B2

公开(公告)日：2022-04-05

申请号：US17112708

申请日：2020-12-04

Applicant: Apple Inc.

Inventor： Can Jin ,  Nicolas Pierre Marie Frederic Bonnier ,  Hao Pan

IPC: G06T19/00 ,  G06F3/01 ,  G06T5/20 ,  G02B27/01

Abstract: A multi-layer low-pass filter is used to filter a first frame of video data representing at least a portion of an environment of an individual. A first layer of the filter has a first filtering resolution setting for a first subset of the first frame, while a second layer of the filter has a second filtering resolution setting for a second subset. The first subset includes a data element positioned along a direction of a gaze of the individual, and the second subset of the frame surrounds the first subset. A result of the filtering is compressed and transmitted via a network to a video processing engine configured to generate a modified visual representation of the environment.

公开(公告)号：US11043018B2

公开(公告)日：2021-06-22

申请号：US16662952

申请日：2019-10-24

Applicant: Apple Inc.

Inventor： Arthur Y Zhang ,  Ray L. Chang ,  Timothy R. Oriol ,  Ling Su ,  Gurjeet S. Saund ,  Guy Cote ,  Jim C. Chou ,  Hao Pan ,  Tobias Eble ,  Avi Bar-Zeev ,  Sheng Zhang ,  Justin A. Hensley ,  Geoffrey Stahl

IPC: G06F3/01 ,  G06T15/00 ,  H04W76/10 ,  G06T3/00 ,  G06T9/00 ,  H04W88/08

Abstract: A mixed reality system that includes a device and a base station that communicate via a wireless connection The device may include sensors that collect information about the user's environment and about the user. The information collected by the sensors may be transmitted to the base station via the wireless connection. The base station renders frames or slices based at least in part on the sensor information received from the device, encodes the frames or slices, and transmits the compressed frames or slices to the device for decoding and display. The base station may provide more computing power than conventional stand-alone systems, and the wireless connection does not tether the device to the base station as in conventional tethered systems. The system may implement methods and apparatus to maintain a target frame rate through the wireless link and to minimize latency in frame rendering, transmittal, and display.

公开(公告)号：US20210127117A1

公开(公告)日：2021-04-29

申请号：US17252701

申请日：2019-06-27

Applicant: Apple Inc.

Inventor： Hao Pan ,  Jim C. Chou ,  Felix C. Fernandes

IPC: H04N19/146 ,  H04N19/14 ,  H04N19/63 ,  H04N19/169 ,  H04N19/176 ,  H04L29/06

Abstract: A video encoding system in which pixel data is decomposed into frequency bands prior to encoding. The frequency bands for a slice of a frame may be buffered so that complexity statistics may be calculated across the frequency bands prior to encoding. The statistics may then be used by a rate control component in determining quantization parameters for the frequency bands for modulating the rate in the encoder for the current slice. The quantization parameters for the frequency bands may be calculated jointly to optimize the quality of the displayed frames after decoder reconstruction and wavelet synthesis on a receiving device. Information about one or more previously processed frames may be used in combination with the statistics for a current slice in determining the quantization parameters for the current slice.

公开(公告)号：US10764588B2

公开(公告)日：2020-09-01

申请号：US16100310

申请日：2018-08-10

Applicant: Apple Inc.

Inventor： Xiang Fu ,  Linfeng Guo ,  Haiyan He ,  Wei Li ,  Xu Gang Zhao ,  Hao Pan ,  Xiaohua Yang ,  Krishnakanth Rapaka ,  Munehiro Nakazato ,  Haitao Guo

IPC: H04N19/176 ,  H04N19/126 ,  G06T7/13 ,  G06N3/08

Abstract: Techniques are disclosed for coding image data adaptively at different levels of downscaling. Such techniques may involve partitioning input data into pixel blocks for coding and performing content analysis on the pixel blocks. The pixel blocks may be input to block coders that operate at different pixel block sizes, which may code the pixel blocks input to them at their respective sizes. Except when a block coder operates at the partitioning size, block coders that operate at different pixel block sizes may perform downscaling of the pixel blocks to match their size with the block coders' respective coding size. A block decoder may invert the coding operations performed by the block coders, decoding coded image data at respective pixel block sizes, then upscaling decoded image data obtained therefrom to a common pixel block size. Image reconstruction may synthesize a resultant image from the decode pixel block data output by the decoders. Quality enhancements may be added by neural network processing systems that have been trained to other segmented images.

公开(公告)号：US20200058152A1

公开(公告)日：2020-02-20

申请号：US16662952

申请日：2019-10-24

Applicant: Apple Inc.

Inventor： Arthur Y. Zhang ,  Ray L. Chang ,  Timothy R. Oriol ,  Ling Su ,  Gurjeet S. Saund ,  Guy Cote ,  Jim C. Chou ,  Hao Pan ,  Tobias Eble ,  Avi Bar-Zeev ,  Sheng Zhang ,  Justin A. Hensley ,  Geoffrey Stahl

IPC: G06T15/00 ,  G06T3/00 ,  G06T9/00 ,  G06F3/01 ,  H04W76/10

Abstract: A mixed reality system that includes a device and a base station that communicate via a wireless connection The device may include sensors that collect information about the user's environment and about the user. The information collected by the sensors may be transmitted to the base station via the wireless connection. The base station renders frames or slices based at least in part on the sensor information received from the device, encodes the frames or slices, and transmits the compressed frames or slices to the device for decoding and display. The base station may provide more computing power than conventional stand-alone systems, and the wireless connection does not tether the device to the base station as in conventional tethered systems. The system may implement methods and apparatus to maintain a target frame rate through the wireless link and to minimize latency in frame rendering, transmittal, and display.

公开(公告)号：US10271054B2

公开(公告)日：2019-04-23

申请号：US14631394

申请日：2015-02-25

Applicant: Apple Inc.

Inventor： Kenneth I. Greenebaum ,  Haitao Guo ,  Hao Pan ,  Guy Cote ,  Andrew Bai

IPC: G06F3/14 ,  G06T5/00 ,  G09G3/20 ,  G09G5/00 ,  G09G5/02 ,  G09G5/10 ,  H04N1/60 ,  H04N1/64 ,  H04N5/20 ,  H04N5/50 ,  H04N9/64 ,  H04N9/67 ,  H04N19/14 ,  H04N19/17 ,  H04N19/30 ,  H04N19/33 ,  H04N19/44 ,  H04N19/52 ,  H04N19/86 ,  H04N19/98 ,  H04N19/136 ,  H04N19/184 ,  H04N19/186 ,  H04N19/169 ,  H04N19/176 ,  H04N19/182 ,  H04N5/235 ,  H04N19/172 ,  H04N19/102 ,  H04N19/137 ,  H04N19/177 ,  H04N21/422 ,  H04N21/431 ,  H04N19/124 ,  H04N19/154 ,  H04N19/463 ,  H04N21/4223 ,  H04N21/44 ,  H04N21/4402 ,  H04N21/442 ,  H04N21/485

Abstract: Adaptive video processing for a target display panel may be implemented in or by a decoding/display pipeline associated with the target display panel. The adaptive video processing methods may take into account video content, display characteristics, and environmental conditions including but not limited to ambient lighting and viewer location when processing and rendering video content for a target display panel in an ambient setting or environment. The display-side adaptive video processing methods may use this information to adjust one or more video processing functions as applied to the video data to render video for the target display panel that is adapted to the display panel according to the ambient viewing conditions.

公开(公告)号：US10212429B2

公开(公告)日：2019-02-19

申请号：US14631401

申请日：2015-02-25

Applicant: Apple Inc.

Inventor： Haitao Guo ,  Hao Pan ,  Guy Cote ,  Andrew Bai

IPC: H04N11/02 ,  H04N19/14 ,  G06T5/00 ,  G09G3/20 ,  G09G5/00 ,  H04N1/60 ,  H04N5/50 ,  H04N9/64 ,  H04N9/67 ,  H04N19/136 ,  H04N19/184 ,  H04N19/186 ,  H04N19/169 ,  H04N19/44 ,  H04N19/30 ,  H04N19/33 ,  H04N19/176 ,  H04N19/182 ,  H04N19/52 ,  G06F3/14 ,  G09G5/02 ,  G09G5/10 ,  H04N1/64 ,  H04N5/20 ,  H04N5/235 ,  H04N19/172 ,  H04N19/102 ,  H04N19/137 ,  H04N19/177 ,  H04N21/422 ,  H04N21/431 ,  H04N19/124 ,  H04N19/154 ,  H04N19/17 ,  H04N19/98 ,  H04N19/86 ,  H04N19/463 ,  H04N21/4223 ,  H04N21/44 ,  H04N21/4402 ,  H04N21/442 ,  H04N21/485

Abstract: Video processing techniques and pipelines that support capture, distribution, and display of high dynamic range (HDR) image data to both HDR-enabled display devices and display devices that do not support HDR imaging. A sensor pipeline may generate standard dynamic range (SDR) data from HDR data captured by a sensor using tone mapping, for example local tone mapping. Information used to generate the SDR data may be provided to a display pipeline as metadata with the generated SDR data. If a target display does not support HDR imaging, the SDR data may be directly rendered by the display pipeline. If the target display does support HDR imaging, then an inverse mapping technique may be applied to the SDR data according to the metadata to render HDR data for display. Information used in performing color gamut mapping may also be provided in the metadata and used to recover clipped colors for display.