公开(公告)号：US20230418372A1

公开(公告)日：2023-12-28

申请号：US18211712

申请日：2023-06-20

Applicant: Apple Inc.

Inventor： Mehmet N. Agaoglu ,  Andrew B. Watson ,  Tim H. Cornelissen ,  Alexander G. Berardino

IPC: G06F3/01

CPC classification number: G06F3/013

Abstract: Various implementations disclosed herein include devices, systems, and methods that determine a gaze behavior state to identify gaze shifting events, gaze holding events, and loss events of a user based on physiological data. For example, an example process may include obtaining eye data associated with a gaze during a first period of time (e.g., eye position and velocity, interpupillary distance, pupil diameters, etc.). The process may further include obtaining head data associated with the gaze during the first period of time (e.g., head position and velocity). The process may further include determining a first gaze behavior state during the first period of time to identify gaze shifting events, gaze holding events, and loss events (e.g., one or more gaze and head pose characteristics may be determined, aggregated, and used to classify the user's eye movement state using machine learning techniques).

公开(公告)号：US10890968B2

公开(公告)日：2021-01-12

申请号：US16376329

申请日：2019-04-05

Applicant: Apple Inc.

Inventor： Yashas Rai Kurlethimar ,  Andrew B. Watson ,  Nicolas P. Bonnier ,  Mehmet N. Agaoglu ,  Alexander G. Berardino ,  Elijah H. Kleeman

IPC: G06F3/01 ,  G02B27/01

Abstract: An electronic device may have a foveated display, an eye-tracking system and a head movement detection system. The eye-tracking system may gather information on a user's point of regard on the display while the head movement detection system may capture information regarding the rotation of the observer's head. Based on the point-of-regard information, head rotation information, image data, the type of eye/head movement that is underway, and/or tiredness information, control circuitry in the electronic device may produce image data for a display, with areas of different resolutions and(or) visual quality. A full-resolution and(or) quality portion of the image may overlap the point of regard. One or more lower resolution portions of the image may surround the full-resolution portion. The control circuitry may include a gaze prediction system for predicting the movement of the user's gaze during a saccade.

公开(公告)号：US20240393876A1

公开(公告)日：2024-11-28

申请号：US18790194

申请日：2024-07-31

Applicant: APPLE INC.

Inventor： Vinay Chawda ,  Mehmet N. Agaoglu ,  Leah M. Gum ,  Paul A. Lacey ,  Julian K. Shutzberg ,  Tim H. Cornelissen ,  Alexander G. Berardino

IPC: G06F3/01 ,  G06F3/04842

Abstract: Various implementations provide views of 3D environments (e.g., extended reality (XR) environments). Non-eye-based user activity, such as hand gestures, is associated with some types of eye-based activity, such as the user gazing at a particular user interface component displayed within a view of a 3D environment. For example, a user's pinching hand gesture may be associated with the user gazing at a particular user interface component, such as a button, at around the same time as the pinching hand gesture is made. These associated behaviors (e.g., the pinch and gaze at the button) may then be interpreted as user input, e.g., user input selecting or otherwise acting upon that user interface component. In some implementations, non-eye-based user activity is only associated with types of eye-based user activity that are likely to correspond to a user perceiving what they are seeing and/or intentionally looking at something.

公开(公告)号：US12099653B2

公开(公告)日：2024-09-24

申请号：US18244570

申请日：2023-09-11

Applicant: Apple Inc.

Inventor： Vinay Chawda ,  Mehmet N. Agaoglu ,  Leah M. Gum ,  Paul A. Lacey ,  Julian K. Shutzberg ,  Tim H. Cornelissen ,  Alexander G. Berardino

IPC: G06F3/01 ,  G06F3/04842

CPC classification number: G06F3/013 ,  G06F3/017 ,  G06F3/04842

Abstract: Various implementations provide views of 3D environments (e.g., extended reality (XR) environments). Non-eye-based user activity, such as hand gestures, is associated with some types of eye-based activity, such as the user gazing at a particular user interface component displayed within a view of a 3D environment. For example, a user's pinching hand gesture may be associated with the user gazing at a particular user interface component, such as a button, at around the same time as the pinching hand gesture is made. These associated behaviors (e.g., the pinch and gaze at the button) may then be interpreted as user input, e.g., user input selecting or otherwise acting upon that user interface component. In some implementations, non-eye-based user activity is only associated with types of eye-based user activity that are likely to correspond to a user perceiving what they are seeing and/or intentionally looking at something.

公开(公告)号：US20250104580A1

公开(公告)日：2025-03-27

申请号：US18828527

申请日：2024-09-09

Applicant: Apple Inc.

Inventor： Stanley K. Melax ,  Seyedpooya Mirhosseini ,  Fuyi Yang ,  Seyedkoosha Mirhosseini ,  Dagny Fleischman ,  David M. Cook ,  Yashas Rai Kurlethimar ,  Xin Wang ,  Travis W. Brown ,  Ara H. Aroyan ,  Jin Wook Chang ,  Abbas Haddadi ,  Yang Li ,  Alexander G. Berardino ,  Mengu Sukan ,  Ermal Dreshaj ,  Kyrollos Yanny ,  William W. Sprague

IPC: G09G3/00 ,  G06F3/01 ,  G06T7/00 ,  G06T11/00

Abstract: Various implementations disclosed herein include devices, systems, and methods that present content items (e.g., movies, TV shows, home-made videos, etc.) on electronic devices such as HMDs. Some implementations adjust what is being displayed by the electronic devices to mitigate optical module-based artifacts (e.g., ghosting). For example, in an HMD with a catadioptric lens, a mirror layer may leak some light to produce ghosting artifacts that may be mitigated by adjusting brightness, dynamic range, contrast, light-spill, color, etc. Some implementations utilize adjustments that are based on content item awareness (e.g., adjustments based on the peak brightness of the scene in a movie that is being displayed within an extended reality (XR) environment, etc.). Some implementations provide adjustments based on environment awareness (e.g., how dark is the surroundings or pass-through environment) and/or optical module modeling. The adjustments may change a particular content item and/or an environment in which it is presented.

公开(公告)号：US20240402800A1

公开(公告)日：2024-12-05

申请号：US18676786

申请日：2024-05-29

Applicant: Apple Inc.

Inventor： Julian K. Shutzberg ,  David J. Meyer ,  David M. Teitelbaum ,  Mehmet N. Agaoglu ,  Ian R. Fasel ,  Chase B. Lortie ,  Daniel J. Brewer ,  Tim H. Cornelissen ,  Leah M. Gum ,  Alexander G. Berardino ,  Lorenzo Soto Doblado ,  Vinay Chawda ,  Itay Bar Yosef ,  Dror Irony ,  Eslam A. Mostafa ,  Guy Engelhard ,  Paul A. Lacey ,  Ashwin Kumar Asoka Kumar Shenoi ,  Bhavin Vinodkumar Nayak ,  Liuhao Ge ,  Lucas Soffer ,  Victor Belyaev ,  Bharat C. Dandu ,  Matthias M. Schroeder ,  Yirong Tang

IPC: G06F3/01 ,  G06F3/04815

Abstract: Various implementations disclosed herein include devices, systems, and methods that interpret user activity as user interactions with user interface (UI) elements positioned within a three-dimensional (3D) space such as an extended reality (XR) environment. Some implementations enable user interactions with virtual elements displayed in 3D environments that utilize alternative input modalities, e.g., XR environments that interpret user activity as either direct interactions or indirect interactions with virtual elements.

公开(公告)号：US20190339770A1

公开(公告)日：2019-11-07

申请号：US16376329

申请日：2019-04-05

Applicant: Apple Inc.

Inventor： Yashas Rai Kurlethimar ,  Andrew B. Watson ,  Nicolas P. Bonnier ,  Mehmet N. Agaoglu ,  Alexander G. Berardino ,  Elijah H. Kleeman

IPC: G06F3/01 ,  G02B27/01

Abstract: An electronic device may have a foveated display, an eye-tracking system and a head movement detection system. The eye-tracking system may gather information on a user's point of regard on the display while the head movement detection system may capture information regarding the rotation of the observer's head. Based on the point-of-regard information, head rotation information, image data, the type of eye/head movement that is underway, and/or tiredness information, control circuitry in the electronic device may produce image data for a display, with areas of different resolutions and(or) visual quality. A full-resolution and(or) quality portion of the image may overlap the point of regard. One or more lower resolution portions of the image may surround the full-resolution portion. The control circuitry may include a gaze prediction system for predicting the movement of the user's gaze during a saccade.