Method of image processing for an augmented reality application

    公开(公告)号:US09947141B2

    公开(公告)日:2018-04-17

    申请号:US15627097

    申请日:2017-06-19

    Applicant: Apple Inc.

    Abstract: An apparatus for and method of image processing in an augmented reality application is provided. The method includes the steps of: providing at least one image of a real environment; performing image processing in an augmented reality application with the at least one image employing visualization of overlaying digital information with visual impressions or the image of the real environment and employing vision-based processing or tracking; and adjusting at least one of a parameter and operating flow of the vision-based processing or tracking depending on at least one of the following: a usage of the image processing, a usage of the visualization, a visually perceivable property of the digital information or the real environment, a property of a display device employed in the visualization, or a manner in which a user is viewing the visualization.

    Method for representing virtual information in a real environment

    公开(公告)号:US09922446B2

    公开(公告)日:2018-03-20

    申请号:US14654395

    申请日:2012-12-21

    Applicant: Apple Inc.

    CPC classification number: G06T15/20 G06T15/06 G06T19/006

    Abstract: A method for representing virtual information in a view of a real environment is provided that includes: providing a system setup including at least one display device, wherein the system setup is adapted for blending in virtual information on the display device in at least part of the view, determining a position and orientation of a viewing point relative to at least one component of the real environment, providing a geometry model of the real environment, providing at least one item of virtual information and a position of the at least one item of virtual information, determining whether the position of the item of virtual information is inside a 2D or 3D geometrical shape, determining a criterion which is indicative of whether the built-in real object is at least partially visible or non-visible in the view of the real environment, and blending in the at least one item of virtual information on the display device in at least part of the view of the real environment.

    Speed adapted touch detection
    33.
    发明授权

    公开(公告)号:US12189888B1

    公开(公告)日:2025-01-07

    申请号:US18482976

    申请日:2023-10-09

    Applicant: Apple Inc.

    Abstract: Detecting a touch includes receiving image data of a touching object of a user selecting selectable objects of a target surface, determining a rate of movement of the touching object, in response to determining that the rate of movement satisfies a predetermined threshold, modifying a touch detection parameter for detecting a touch event between the touching object and the target surface, and detecting one or more additional touch events using the modified touch detection parameter.

    Method and System for Providing At Least One Image Captured By a Scene Camera of a Vehicle

    公开(公告)号:US20230328207A1

    公开(公告)日:2023-10-12

    申请号:US18333943

    申请日:2023-06-13

    Applicant: Apple Inc.

    CPC classification number: H04N7/181 G06F3/013 G06V40/19 B60R11/04 G06V20/56

    Abstract: The present disclosure relates to a method of providing at least one image of at least one real object captured by at least one scene camera of a plurality of scene cameras mounted on a vehicle. The method includes: providing camera poses of respective scene cameras of the plurality of scene cameras relative to a reference coordinate system associated with the vehicle, providing user attention data related to a user captured by an information capturing device, providing at least one attention direction relative to the reference coordinate system from the user attention data, determining at least one of the scene cameras among the plurality of scene cameras according to the at least one attention direction and the respective camera pose of the at least one of the scene cameras, and providing at least one image of at least one real object captured by the at least one of the scene cameras.

    IMU for Touch Detection
    35.
    发明申请

    公开(公告)号:US20210318748A1

    公开(公告)日:2021-10-14

    申请号:US17355320

    申请日:2021-06-23

    Applicant: Apple Inc.

    Abstract: Touch detection may include determining, based on data from an IMU on a first device that monitors movement of a touching object, a touch event, wherein the touch event indicates contact between the touching object and a surface, obtaining a depth image captured by a second device, comprising a touch image, determining a touch point of the object based on the touch image, and providing a response based on the touch point of the object and the touched surface.

    Generating a 3D Model of a Fingertip for Visual Touch Detection

    公开(公告)号:US20210209840A1

    公开(公告)日:2021-07-08

    申请号:US17205830

    申请日:2021-03-18

    Applicant: Apple Inc.

    Abstract: Generating a 3D model may include determining, based on sensor data from a touch sensor on a first device, a touch event, wherein the touch event comprises a touch on the first device by a touching object. Generating a 3D model may also include, in response to a touch event, obtaining a first image of the touching object by a first camera of the first device, and obtaining, from a second device, a second image of the touching object, wherein the first image of the touching object captures a first view of the touching object, and wherein the second image of the touching object captures a second view of the touching object. A model of the touching object is generated based on the first image and the second image.

    Method for Representing Virtual Information in a Real Environment

    公开(公告)号:US20210183136A1

    公开(公告)日:2021-06-17

    申请号:US17129340

    申请日:2020-12-21

    Applicant: Apple Inc.

    Abstract: A method for representing virtual information in a view of a real environment is provided that includes: providing a system setup including at least one display device, wherein the system setup is adapted for blending in virtual information on the display device in at least part of the view, determining a position and orientation of a viewing point relative to at least one component of the real environment, providing a geometry model of the real environment, providing at least one item of virtual information and a position of the at least one item of virtual information, determining whether the position of the item of virtual information is inside a 2D or 3D geometrical shape, determining a criterion which is indicative of whether the built-in real object is at least partially visible or non-visible in the view of the real environment, and blending in the at least one item of virtual information on the display device in at least part of the view of the real environment.

    Finger Orientation Touch Detection
    38.
    发明申请

    公开(公告)号:US20210096652A1

    公开(公告)日:2021-04-01

    申请号:US17022583

    申请日:2020-09-16

    Applicant: Apple Inc.

    Abstract: Touch detection includes obtaining image data for a finger and a target surface, and determining a touch region in the image data on which the fingertip makes contact with the target surface. A pointing direction of the finger in the image data is determined, and a target region is estimated on the target surface based on the touch region and the pointing direction, wherein the touch region comprises a portion of the target surface that extends from the touch region in a direction of the pointing direction.

    Method and system for providing position or movement information for controlling at least one function of an environment

    公开(公告)号:US10937187B2

    公开(公告)日:2021-03-02

    申请号:US16371519

    申请日:2019-04-01

    Applicant: Apple Inc.

    Abstract: A method for providing position information for controlling at least one function of an environment provided with an environment coordinate system comprises the steps of receiving image information associated with at least one image of at least one part of the environment captured by a camera of a mobile device, the mobile device being separate from the environment, determining a position associated with the mobile device in the environment coordinate system according to the image information associated with the at least one image, and providing position information indicative of the position associated with the mobile device in the environment coordinate system to control at least one function of the environment according to the position. There is also provided a method and system which provides movement information indicative of a movement associated with the mobile device relative to the environment coordinate system to control at least one function of the environment according to the movement.

    Depth-based touch detection
    40.
    发明授权

    公开(公告)号:US10572072B2

    公开(公告)日:2020-02-25

    申请号:US15721207

    申请日:2017-09-29

    Applicant: Apple Inc.

    Abstract: Systems, methods, and computer readable media to improve the operation of detecting contact between a finger or other object and a surface are described. In general, techniques disclosed herein utilize a depth map to identify an object and a surface, and a classifier to determine when the object is touching the surface. Unlike the prior art, a measure of the object's “distance” is made relative to the surface and not the camera(s) thereby providing some measure of invariance with respect to camera pose. The object-surface distance measure can be used to construct an identifier or “feature vector” that, when applied to a classifier, generates an output indicative of whether the object is touching the surface. The classifier may be based on machine learning and can be trained off-line before run-time operations are commenced. In some embodiments, temporal filtering may be used to improve surface detection operations.

Patent Agency Ranking