Invention Grant
- Patent Title: Multi-modal lie detection method and apparatus, and device
-
Application No.: US17389383Application Date: 2021-07-30
-
Publication No.: US11244119B1Publication Date: 2022-02-08
- Inventor: Jianhua Tao , Licai Sun , Bin Liu , Zheng Lian
- Applicant: Institute of Automation, Chinese Academy of Sciences
- Applicant Address: CN Beijing
- Assignee: Institute of Automation, Chinese Academy of Sciences
- Current Assignee: Institute of Automation, Chinese Academy of Sciences
- Current Assignee Address: CN Beijing
- Agency: Bayramoglu Law Offices LLC
- Priority: CN202110001041.2 20210104
- Main IPC: G06F40/35
- IPC: G06F40/35 ; G10L15/08 ; G10L15/02 ; G10L15/24 ; G06N3/04 ; G06K9/62 ; G06K9/00

Abstract:
A multi-modal lie detection method and apparatus, and a device to improve an accuracy of an automatic lie detection are provided. The multi-modal lie detection method includes inputting original data of three modalities, namely a to-be-detected audio, a to-be-detected video and a to-be-detected text; performing a feature extraction on input contents to obtain deep features of the three modalities; explicitly depicting first-order, second-order and third-order interactive relationships of the deep features of the three modalities to obtain an integrated multi-modal feature of each word; performing a context modeling on the integrated multi-modal feature of the each word to obtain a final feature of the each word; and pooling the final feature of the each word to obtain global features, and then obtaining a lie classification result by a fully-connected layer.
Information query