Methods and systems for multimodal hand state prediction

    公开(公告)号:US11782522B1

    公开(公告)日:2023-10-10

    申请号:US17704970

    申请日:2022-03-25

    摘要: In various examples, the present disclosure describes methods and systems for generating hand state predictions. A hand state prediction system includes a machine learning-based model, such as a neural network model, that is trained to convert inertial motion measurements and surface contact data into predictions of a corresponding hand gesture or gripping posture. For each window of data sampled, motion data and contact data are obtained, processed and fused to generate a fused prediction. The hand state prediction system can operate in continuous mode to automatically detect a start and an end of a hand action, or a user can designate a start and an end of a hand action. A hand state prediction is generated by a multimodal classifier by processing the fused prediction. Instructions represented by a hand action can be acted upon through a command action performed by a computing device or a computer application.

    METHODS AND SYSTEMS FOR MULTIMODAL HAND STATE PREDICTION

    公开(公告)号:US20230305644A1

    公开(公告)日:2023-09-28

    申请号:US17704970

    申请日:2022-03-25

    摘要: In various examples, the present disclosure describes methods and systems for generating hand state predictions. A hand state prediction system includes a machine learning-based model, such as a neural network model, that is trained to convert inertial motion measurements and surface contact data into predictions of a corresponding hand gesture or gripping posture. For each window of data sampled, motion data and contact data are obtained, processed and fused to generate a fused prediction. The hand state prediction system can operate in continuous mode to automatically detect a start and an end of a hand action, or a user can designate a start and an end of a hand action. A hand state prediction is generated by a multimodal classifier by processing the fused prediction. Instructions represented by a hand action can be acted upon through a command action performed by a computing device or a computer application.