公开(公告)号：US11915097B1

公开(公告)日：2024-02-27

申请号：US17143411

申请日：2021-01-07

Applicant: Apple Inc.

Inventor： Mohamed Selim Ben Himane ,  Anselm Grundhoefer ,  Arun Srivatsan Rangaprasad ,  Jeffrey S. Norris ,  Paul Ewers ,  Scott G. Wade ,  Thomas G. Salter ,  Tom Sengelaub

IPC: G06K9/18 ,  G06K7/14 ,  G06T7/90 ,  G06N20/00

CPC classification number: G06K7/1443 ,  G06K7/1417 ,  G06K7/1456 ,  G06N20/00 ,  G06T7/90

Abstract: Various implementations disclosed herein include devices, systems, and methods that provide color visual markers that include colored markings that encode data, where the colors of the colored markings are determined by scanning (e.g., detecting the visual marker using a sensor of an electronic device) the visual marker itself. In some implementations, a visual marker is detected in an image of a physical environment. In some implementations, the visual marker is detected in the image by detecting a predefined shape of a first portion of the visual marker in the image. Then, a color-interpretation scheme is determined for interpreting colored markings of the visual marker that encode data by identifying a set of colors at a corresponding set of predetermined locations on the visual marker. Then, the data of the visual marker is decoded using the colored markings and the set of colors of the color-interpretation scheme.

公开(公告)号：US20230377302A1

公开(公告)日：2023-11-23

申请号：US18027910

申请日：2021-09-12

Applicant: Apple Inc.

Inventor： Martin Haller ,  Harihar Narasimha-Iyer ,  Tom Sengelaub

IPC: G06V10/141 ,  G06T7/70 ,  G06T7/00 ,  G06F3/01 ,  G06F21/32

CPC classification number: G06V10/141 ,  G06T7/70 ,  G06T7/0002 ,  G06F3/013 ,  G06F21/32 ,  G06T2207/30168 ,  G06T2207/30201

Abstract: Methods and apparatus for flexible illumination that improve the performance and robustness of an imaging system are described. Multiple different lighting configurations for the imaging system are pre-generated. Each lighting configuration may specify one or more aspects of lighting. A lookup table may be generated via which each pose is associated with a respective lighting configuration. A user may put on, hold, or otherwise use the device. A process may be initiated in which different lighting configurations may be selected by the controller to capture images of the user's eye, periorbital region, or face at different poses and in different conditions for use by a biometric authentication or gaze tracking process.

公开(公告)号：US11710350B2

公开(公告)日：2023-07-25

申请号：US17499205

申请日：2021-10-12

Applicant: Apple Inc.

Inventor： Tom Sengelaub ,  Hua Gao ,  Hao Qin ,  Julia Benndorf

IPC: G06V40/19 ,  G06F3/01 ,  G06T7/73 ,  G06T7/50 ,  G06T7/90 ,  G06T7/70 ,  G06V40/16

CPC classification number: G06V40/19 ,  G06F3/013 ,  G06T7/50 ,  G06T7/70 ,  G06T7/74 ,  G06T7/90 ,  G06V40/165 ,  G06T2200/04 ,  G06T2207/10024 ,  G06T2207/10028 ,  G06T2207/10048 ,  G06T2207/30041 ,  G06T2207/30201

Abstract: Some implementations of the disclosure involve, at a device having one or more processors, one or more image sensors, and an illumination source, detecting a first attribute of an eye based on pixel differences associated with different wavelengths of light in a first image of the eye. These implementations next determine a first location associated with the first attribute in a three dimensional (3D) coordinate system based on depth information from a depth sensor. Various implementations detect a second attribute of the eye based on a glint resulting from light of the illumination source reflecting off a cornea of the eye. These implementations next determine a second location associated with the second attribute in the 3D coordinate system based on the depth information from the depth sensor, and determine a gaze direction in the 3D coordinate system based on the first location and the second location.

公开(公告)号：US11676302B2

公开(公告)日：2023-06-13

申请号：US16878932

申请日：2020-05-20

Applicant: APPLE INC.

Inventor： Jan Hoffmann ,  Tom Sengelaub ,  Denis Williams

IPC: G06K9/00 ,  G06T7/73 ,  A61B3/113 ,  G06V40/19

CPC classification number: G06T7/75 ,  A61B3/113 ,  G06V40/19 ,  G06T2200/04 ,  G06T2207/10028 ,  G06T2207/20076 ,  G06T2207/30201

Abstract: A system for determining the gaze endpoint of a subject, the system comprising: a eye tracking unit adapted to determine the gaze direction of one or more eyes of the subject; a head tracking unit adapted to determine the position comprising location and orientation of the eye tracker with respect to a reference coordinate system; a 3D Structure representation unit, that uses the 3D structure and position of objects of the scene in the reference coordinate system to provide a 3D structure representation of the scene; based on the gaze direction, the eye tracker position and the 3D structure representation, calculating the gaze endpoint on an object of the 3D structure representation of the scene or determining the object itself.

公开(公告)号：US20210097714A1

公开(公告)日：2021-04-01

申请号：US17032227

申请日：2020-09-25

Applicant: Apple Inc.

Inventor： Anselm Grundhoefer ,  Jeffrey S. Norris ,  Mohamed Selim Ben Himane ,  Paul Ewers ,  Scott G. Wade ,  Shih-Sang (Carnaven) Chiu ,  Thomas G. Salter ,  Tom Sengelaub ,  Viral N. Parekh

IPC: G06T7/73 ,  G06T7/536 ,  G06T7/60 ,  G06K7/14

Abstract: Various implementations disclosed herein include devices, systems, and methods that determine the relative positioning (e.g., offset) between a mobile electronic device and a visual marker. In some implementations, the determined relative positioning and a known position of the visual marker are used to determine a position (e.g., geo coordinates) of the mobile electronic device that is more accurate than existing techniques. In some implementations, the determined relative positioning is used with a position of the mobile electronic device to crowd source the stored position of the visual marker. In some implementations, the determined relative positioning and a position of the visual marker are used to determine a position of an object detected in an image by the mobile electronic device. In some implementations at an electronic device having a processor, locally-determined locations of a visual marker are received from mobile electronic devices that scan a visual marker.

公开(公告)号：US10546193B2

公开(公告)日：2020-01-28

申请号：US15750032

申请日：2016-08-05

Applicant: APPLE INC.

Inventor： Eberhard Schmidt ,  Tom Sengelaub

IPC: G06K9/00 ,  A61B5/16 ,  G05B19/042 ,  G06T7/70 ,  A61B3/113 ,  A61B5/11 ,  G06Q10/06

Abstract: The invention relates to a system and method to control a workflow comprising at least one task to be performed by a person (P3), wherein information is provided about at least one certain object (20, 32, 36, 38) related to the at least one task of the workflow, eye data (24, 26) are captured of at least one eye of the person (P3), and in dependency of the eye data (24, 26) and the information about the at least one certain object (20, 32, 36, 38) it is checked whether at least one task condition consisting in whether the task had been performed and/or whether the task is allowed to be performed is fulfilled. The invention also relates to a system and method for providing a set of task-specific control parameters (CP).

公开(公告)号：US10437327B2

公开(公告)日：2019-10-08

申请号：US15571966

申请日：2016-05-04

Applicant: APPLE INC.

Inventor： Tom Sengelaub ,  Martin Haller ,  Jan Hoffmann

IPC: G06K9/00 ,  G06F3/01 ,  A61B3/113

Abstract: The invention relates to an eye tracking device (10a; 10b; 10c) comprising a processing unit (18) and an optical system (14), which comprises a capturing unit. The optical system (14) provides a first optical path (P; P1) with a first imaging property and a second optical path (P; P2) with a second imaging property and the capturing unit (C; C1, C2) captures a first image (24a) by capturing light that has passed along the first optical path (P; P1) and a second image (24b) by capturing light that has passed along the second optical path (P; P2), so that due to the difference of the first and second imaging property the first (24a) and the second image (24b) comprise a difference related to a characteristic of at least part of the first and the second image (24b), wherein the eye tracking device (10a; 10b; 10c) is configured to determine a property of the eye (12) on the basis of at least one of the first image (24a) and the second image (24b).

公开(公告)号：US20250104669A1

公开(公告)日：2025-03-27

申请号：US18907280

申请日：2024-10-04

Applicant: Apple Inc.

Inventor： Ritu Shah ,  Ryan J. Dunn ,  Tom Sengelaub

IPC: G09G5/10 ,  G06F3/01 ,  G06T7/90 ,  G06V10/25 ,  G09G3/34

Abstract: Operating an outward-facing display device for low light conditions may include receiving information indicative of ambient lighting conditions for the display device's environment; in accordance with a determination that the ambient lighting conditions do not satisfy a brightness criterion, determining a set of pixels in the outward-facing display device corresponding to an area of interest in the environment; and driving the set of pixels at a particular brightness to improve lighting in the area of interest.

公开(公告)号：US20240211038A1

公开(公告)日：2024-06-27

申请号：US18534388

申请日：2023-12-08

Applicant: Apple Inc.

Inventor： Hao Qin ,  Hua Gao ,  Tom Sengelaub

IPC: G06F3/01 ,  G02B27/01 ,  G06T19/00

CPC classification number: G06F3/013 ,  G02B27/017 ,  G06F3/017 ,  G06T19/006

Abstract: Methods to trigger eye enrollment without requiring good gaze interaction are described that allow a guest user of a device to initiate partial or full eye enrollment even though their eye model is not known and thus conventional gaze-based interactions do not work well. A gaze tracking system collects gaze data in the background. At any time (or within an interval after a user puts on the device), an eye enrollment can be triggered by detecting some gaze gesture, for example rolling the eyes in a large circle, or moving the eyes randomly for a time that exceeds a threshold. Depending on the coverage of the gaze/cornea data collected in the background, a full eye enrollment or only a visual axis enrollment may be performed in response to the gesture.

公开(公告)号：US11948332B2

公开(公告)日：2024-04-02

申请号：US18111099

申请日：2023-02-17

Applicant: APPLE INC.

Inventor： Jan Hoffmann ,  Tom Sengelaub ,  Denis Williams

IPC: G06K9/00 ,  A61B3/113 ,  G06T7/73 ,  G06V40/19

CPC classification number: G06T7/75 ,  A61B3/113 ,  G06V40/19 ,  G06T2200/04 ,  G06T2207/10028 ,  G06T2207/20076 ,  G06T2207/30201

Abstract: A system for determining the gaze endpoint of a subject, the system comprising: a eye tracking unit adapted to determine the gaze direction of one or more eyes of the subject; a head tracking unit adapted to determine the position comprising location and orientation of the eye tracker with respect to a reference coordinate system; a 3D Structure representation unit, that uses the 3D structure and position of objects of the scene in the reference coordinate system to provide a 3D structure representation of the scene; based on the gaze direction, the eye tracker position and the 3D structure representation, calculating the gaze endpoint on an object of the 3D structure representation of the scene or determining the object itself.