Invention Grant
- Patent Title: Method and system for generating 2D animated lip images synchronizing to an audio signal
-
Application No.: US17405765Application Date: 2021-08-18
-
Publication No.: US11887238B2Publication Date: 2024-01-30
- Inventor: Swapna Agarwal , Dipanjan Das , Brojeshwar Bhowmick
- Applicant: Tata Consultancy Services Limited
- Applicant Address: IN Mumbai
- Assignee: TATA CONSULTANCY SERVICES LIMITED
- Current Assignee: TATA CONSULTANCY SERVICES LIMITED
- Current Assignee Address: IN Mumbai
- Agency: FINNEGAN, HENDERSON, FARABOW, GARRETT & DUNNER, LLP
- Priority: IN 2021036263 2020.08.23
- Main IPC: G06T13/80
- IPC: G06T13/80 ; G06V40/16 ; G06V10/40 ; G10L15/02 ; G10L19/00 ; G06F18/214 ; G06F18/25

Abstract:
A method and system for generating 2D animated lip images synchronizing to an audio signal for an unseen subject. The system receives an audio signal and a target lip image of an unseen target subject as inputs from a user and processes these inputs to extract a plurality of high dimensional audio image features. The lip generator system is meta-trained with training dataset which consists of large variety of subjects' ethnicity and vocabulary. The meta-trained model generates realistic animation for previously unseen face and unseen audio when finetuned with only a few-shot samples for a predefined interval of time. Additionally, the method protects intrinsic features of the unseen target subject.
Public/Granted literature
- US20220058850A1 METHOD AND SYSTEM FOR GENERATING 2D ANIMATED LIP IMAGES SYNCHRONIZING TO AN AUDIO SIGNAL Public/Granted day:2022-02-24
Information query