公开(公告)号：US20210132681A1

公开(公告)日：2021-05-06

申请号：US16475384

申请日：2018-05-23

Applicant: SOUTHEAST UNIVERSITY

Inventor： Aiguo SONG ,  Hui ZHANG ,  Yuqing YU ,  Huanhuan QIN ,  Huijun LI ,  Baoguo XU

IPC: G06F3/01

Abstract: A natural human-computer interaction system based on multi-sensing data fusion comprises a MEMS anti tracking device, a visual tracking device, a force feedback device and a PC terminal. The MEMS aim tracking device is composed of three sets of independent MEMS sensors for collecting arm joint angle information and measuring an arm motion trajectory. The visual tracking device is composed of a binocular camera for collecting image information and measuring a finger motion trajectory. The force feedback device is mounted in a palm of an operator for providing a feedback force to the finger. The PC terminal comprises a data display module, an arm motion calculating module, an image processing module, a mechanics calculating module and a virtual scene rendering module. The system tracks the arm motion trajectory of the operator by taking and tracks the finger motion trajectory of the operator and provides force feedback interaction to the finger of the operator.