Behavior-based standard operating procedure detection

    公开(公告)号:US12073027B2

    公开(公告)日:2024-08-27

    申请号:US18084885

    申请日:2022-12-20

    摘要: Implementations are directed to receiving a first set of images included in a first video captured by a camera that monitors a human performing a task; processing the first set of images using a first machine learning (ML) model to determine whether the first set of images depicts a gesture that is included in a predefined set of gestures; in response to determining that the first set of images depicts a gesture included in a predefined set of gestures, processing a second set of images included in the first video using a second ML model to determine a first gesture type of the gesture; comparing the first gesture type with a first expected gesture type to determine whether performance of the task conforms to a standard operating procedure (SOP) for the task; and providing feedback representative of a comparison result in a user interface.

    Devices, Methods, and Graphical User Interfaces for Interacting with Three-Dimensional Environments

    公开(公告)号:US20240282060A1

    公开(公告)日:2024-08-22

    申请号:US18584911

    申请日:2024-02-22

    申请人: Apple Inc.

    IPC分类号: G06T19/00 G06F3/01 G06T19/20

    摘要: A computer system displays a first view of a three-dimensional environment, including a first user interface object, that when activated by a user input meeting first criteria, causes performance of a first operation. While displaying the first view, the computer system detects first movement of a hand in a physical environment, and in response, changes an appearance of the first user interface object in the first view based on the first movement, including: in accordance with a determination that the first movement meets the first criteria requiring the hand move in a first manner, performing and indicating performance of the first operation; and in accordance with a determination that the first movement does not meet the first criteria, moving the first user interface object away from a position in the three-dimensional environment corresponding to a location of the hand in the physical environment without performing the first operation.

    Simultaneous Controller and Touch Interactions

    公开(公告)号:US20240281070A1

    公开(公告)日:2024-08-22

    申请号:US18310622

    申请日:2023-05-02

    IPC分类号: G06F3/01 G06F3/0346 G06V40/20

    摘要: An artificial reality (XR) system includes XR controllers and hand gestures and corresponding representation of the hand gestures in XR space. Example implementations allow a user to use controllers for distance interaction and interactions that require controller buttons, but also recognize when the user extends her finger and wants to manually touch things-using touch in conjunction with controllers. Thus, if the user has a controller in her hand and wants to touch a UI element, she can also use her finger. In particular, the system identifies the finger is not touching the controller (e.g., via capacitance sensors) and that a hand pose is present (e.g., finger extended—from camera), thereby processing hand input from the finger without the user having to put down the controller.

    System and method for matching a test frame sequence with a reference frame sequence

    公开(公告)号:US12067664B2

    公开(公告)日:2024-08-20

    申请号:US17653057

    申请日:2022-03-01

    申请人: Lindera GMBH

    摘要: A pose data file may represent, for each frame of a reference frame sequence, a plurality of two-dimensional skeleton projections on a virtual spherical surface, each of which, for a particular frame, corresponds to a two-dimensional reference pose image of a three-dimensional skeleton of a first human from a viewing angle. A real-time two-dimensional skeleton detector module detects a two-dimensional skeleton of a second human in each received test frame of a test frame sequence. A pose matching module selects a particular two-dimensional skeleton projection of the first human with the minimum mathematical distance from the two-dimensional skeleton of the second human in the current test frame to match the current pose of the second human in the current test frame with a corresponding reference pose image of the pose data file. The particular two-dimensional skeleton projection represents the corresponding reference pose image at the viewing angle.