公开(公告)号：US12206830B2

公开(公告)日：2025-01-21

申请号：US18661336

申请日：2024-05-10

Applicant: Apple Inc.

Inventor： Alexander Menzies ,  Tobias Rick ,  Alexandre Da Veiga ,  Bryce L Schmidtchen ,  Vedant Saran ,  Bryan Cline ,  Michael I Weinstein ,  Tsao-Wei Huang

IPC: G06T5/00 ,  G06T5/50 ,  G06T11/60 ,  H04N13/20

Abstract: Embodiments disclosed herein are directed to devices, systems, and methods for separately processing an image stream to generate a first set of images for display by a set of displays and a second set of images for storage or transfer as part of a media capture event. Specifically, a first set of images from an image stream may be used to generate a first set of transformed images, and these transformed images may be displayed on a set of displays. A second set of images may be selected from the image stream in response to a capture request associated with a media capture event, and the second set of images may be used to generate a second set of transformed images. A set of output images may be generated from the second set of transformed images, and the set of output images may be stored or transmitted for later viewing.

公开(公告)号：US20240355081A1

公开(公告)日：2024-10-24

申请号：US18760288

申请日：2024-07-01

Applicant: Apple Inc.

Inventor： Bryce L. Schmidtchen ,  Bryan Cline ,  Charles O. Goddard ,  Michael I. Weinstein ,  Tsao-Wei Huang ,  Tobias Rick ,  Vedant Saran ,  Alexander Menzies ,  Alexandre Da Veiga

IPC: G06T19/20 ,  G06T5/70 ,  G06T15/04 ,  G06T15/10 ,  G06T19/00

CPC classification number: G06T19/20 ,  G06T5/70 ,  G06T15/04 ,  G06T15/10 ,  G06T19/006 ,  G06T2219/2004

Abstract: Various implementations provide a view of a 3D environment including a portal for viewing a stereo item (e.g., a photo or video) positioned a distance behind the portal. One or more visual effects are provided based on texture of one or more portions of the stereo item, e.g., texture at cutoff or visible edges of the stereo item. The effects change the appearance of the stereo item or the portal itself, e.g., improving visual comfort issues by minimizing window violations or otherwise enhancing the viewing experience. Various implementations provide a view of a 3D environment including an immersive view of a stereo item without using portal. Such a visual effect may be provided to partially obscure the surrounding 3D environment.

公开(公告)号：US20240339027A1

公开(公告)日：2024-10-10

申请号：US18744965

申请日：2024-06-17

Applicant: APPLE INC.

Inventor： Jeffrey S. Norris ,  Bruno M. Sommer ,  Alexandre Da Veiga

IPC: G08C17/02 ,  G06F18/22 ,  G06V10/22 ,  H04W8/00 ,  H04W76/10

CPC classification number: G08C17/02 ,  G06F18/22 ,  G06V10/22 ,  H04W8/005 ,  H04W76/10 ,  G08C2201/31

Abstract: A method includes receiving a signal that indicates a location of a control device that is configured to change an operating state of a controlled device. The method also includes identifying a first visible device and a second visible device in one or more images, matching the first visible device with the control device based on a location of the first visible device matching the location of the control device, matching the second visible device with the controlled device, pairing the control device with a host device, and controlling the control device using the host device to change the operating state of the controlled device.

公开(公告)号：US12039859B2

公开(公告)日：2024-07-16

申请号：US17980057

申请日：2022-11-03

Applicant: APPLE INC.

Inventor： Jeffrey S. Norris ,  Bruno M. Sommer ,  Alexandre Da Veiga

IPC: H04W76/10 ,  G06F18/22 ,  G06V10/22 ,  G08C17/02 ,  H04W8/00

CPC classification number: G08C17/02 ,  G06F18/22 ,  G06V10/22 ,  H04W8/005 ,  H04W76/10 ,  G08C2201/31

Abstract: A method includes receiving one or more signals that each indicate a device type for a respective remote device, identifying one or more visible devices in one or more images, matching a first device from the one or more visible devices with a first signal from the one or more signals based on a device type of the first device matching a device type for the first signal and based on a visible output of the first device, pairing the first device with a second device, and controlling a function of the first device using the second device.

公开(公告)号：US11941818B1

公开(公告)日：2024-03-26

申请号：US17190969

申请日：2021-03-03

Applicant: Apple Inc.

Inventor： Vedant Saran ,  Alexandre Da Veiga

IPC: G06T7/13 ,  G06T7/50 ,  G06T7/70

CPC classification number: G06T7/13 ,  G06T7/50 ,  G06T7/70

Abstract: Various implementations disclosed herein include devices, systems, and methods that determine a 3D location of an edge based on image and depth data. This involves determining a 2D location of a line segment corresponding to an edge of an object based on a light-intensity image, determining a 3D location of a plane based on depth values (e.g., based on sampling depth near the edge/on both sides of the edge and fitting a plane to the sampled points), and determining a 3D location of the line segment based on the plane (e.g., by projecting the line segment onto the plane). The devices, systems, and methods may involve classifying an edge as a particular edge type (e.g., fold, cliff, plane) and detecting the edge based on such classification.

公开(公告)号：US20240062413A1

公开(公告)日：2024-02-22

申请号：US18384352

申请日：2023-10-26

Applicant: Apple Inc.

Inventor： Jeffrey S. Norris ,  Alexandre Da Veiga ,  Bruno M. Sommer ,  Ye Cong ,  Tobias Eble ,  Moinul Khan ,  Nicolas Bonnier ,  Hao Pan

IPC: G06T7/73 ,  G02B27/01 ,  G06V20/20 ,  G06V10/46 ,  G06V10/24

CPC classification number: G06T7/73 ,  G02B27/017 ,  G06V20/20 ,  G06V10/462 ,  G06V10/245 ,  G06T2207/30204

Abstract: A method includes obtaining first pass-through image data characterized by a first pose. The method includes obtaining respective pixel characterization vectors for pixels in the first pass-through image data. The method includes identifying a feature of an object within the first pass-through image data in accordance with a determination that pixel characterization vectors for the feature satisfy a feature confidence threshold. The method includes displaying the first pass-through image data and an AR display marker that corresponds to the feature. The method includes obtaining second pass-through image data characterized by a second pose. The method includes transforming the AR display marker to a position associated with the second pose in order to track the feature. The method includes displaying the second pass-through image data and maintaining display of the AR display marker that corresponds to the feature of the object based on the transformation.

公开(公告)号：US20230403386A1

公开(公告)日：2023-12-14

申请号：US18238750

申请日：2023-08-28

Applicant: Apple Inc.

Inventor： Alexandre Da Veiga ,  Tobias Rick

IPC: H04N13/282 ,  H04N13/383 ,  H04N13/128 ,  H04N13/261

CPC classification number: H04N13/282 ,  H04N13/383 ,  H04N13/128 ,  H04N13/261 ,  H04N13/332

Abstract: Various implementations disclosed herein include devices, systems, and methods that provides a view of a three-dimensional (3D) environment that includes a projection of a 3D image, such as a multi-directional stereo image or video content. For example, an example process may include obtaining a three-dimensional (3D) image including a stereoscopic image pair including left eye content corresponding to a left eye viewpoint and right eye content corresponding to a right eye viewpoint, generating a projection of the 3D image within a 3D environment by projecting portions of the 3D image to form a shape within the 3D environment, the shape based on an angle of view of the 3D image, where the 3D environment includes additional content separate from the 3D image, and providing a view of the 3D environment including the projection of the 3D image.

公开(公告)号：US20230344973A1

公开(公告)日：2023-10-26

申请号：US18215371

申请日：2023-06-28

Applicant: Apple Inc.

Inventor： Shai Messingher Lang ,  Alexandre Da Veiga ,  Spencer H. Ray ,  Symeon Delikaris Manias

IPC: G11B27/10 ,  G10L25/51 ,  H04N9/802 ,  H04S7/00

CPC classification number: H04N9/802 ,  G10L25/51 ,  G11B27/10 ,  H04S7/302

Abstract: Various implementations disclosed herein include devices, systems, and methods that that modify audio of played back AV content based on context in accordance with some implementations. In some implementations audio-visual content of a physical environment is obtained, and the audio-visual content includes visual content and audio content that includes a plurality of audio portions corresponding to the visual content. In some implementations, a context for presenting the audio-visual content is determined, and a temporal relationship between one or more audio portions of the plurality of audio portions and the visual content is determined based on the context. Then, synthesized audio-visual content is presented based on the temporal relationship.

公开(公告)号：US11763479B2

公开(公告)日：2023-09-19

申请号：US18072855

申请日：2022-12-01

Applicant: Apple Inc.

Inventor： Amit Jain ,  Aditya Sankar ,  Qi Shan ,  Alexandre Da Veiga ,  Shreyas V. Joshi

IPC: G06T7/62 ,  G06T7/60 ,  G06V20/64 ,  G06F18/241 ,  G06V10/764 ,  G06V10/80 ,  G06V10/82 ,  G06V10/70 ,  G06V10/44 ,  G06V20/00 ,  G06V10/22

CPC classification number: G06T7/62 ,  G06F18/241 ,  G06T7/60 ,  G06V10/22 ,  G06V10/454 ,  G06V10/764 ,  G06V10/803 ,  G06V10/82 ,  G06V10/87 ,  G06V20/00 ,  G06V20/64 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Various implementations disclosed herein include devices, systems, and methods that provide measurements of objects based on a location of a surface of the objects. An exemplary process may include obtaining a three-dimensional (3D) representation of a physical environment that was generated based on depth data and light intensity image data, generating a 3D bounding box corresponding to an object in the physical environment based on the 3D representation, determining a class of the object based on the 3D semantic data, determining a location of a surface of the object based on the class of the object, the location determined by identifying a plane within the 3D bounding box having semantics in the 3D semantic data satisfying surface criteria for the object, and providing a measurement of the object, the measurement of the object determined based on the location of the surface of the object.

公开(公告)号：US11756229B2

公开(公告)日：2023-09-12

申请号：US17493359

申请日：2021-10-04

Applicant: Apple Inc.

Inventor： Bruno M. Sommer ,  Alexandre Da Veiga

IPC: G06T7/73 ,  G06T7/20 ,  G06F18/214

CPC classification number: G06T7/73 ,  G06F18/2148 ,  G06T7/20

Abstract: Systems and methods for localization for mobile devices are described. Some implementations may include accessing motion data captured using one or more motion sensors; determining, based on the motion data, a coarse localization, wherein the coarse localization includes a first estimate of position; obtaining one or more feature point maps, wherein the feature point maps are associated with a position of the coarse localization; accessing images captured using one or more image sensors; determining, based on the images, a fine localization pose by localizing into a feature point map of the one or more feature point maps, wherein the fine localization pose includes a second estimate of position and an estimate of orientation; generating, based on the fine localization pose, a virtual object image including a view of a virtual object; and displaying the virtual object image.