Invention Publication
- Patent Title: METHOD AND SYSTEM FOR LATENT-SPACE FACIAL FEATURE EDITING IN DEEP LEARNING BASED FACE SWAPPING
-
Application No.: US17707782Application Date: 2022-03-29
-
Publication No.: US20230316587A1Publication Date: 2023-10-05
- Inventor: Sirak Ghebremusse , Stéphane Grabli , Jacek Krzysztof Naruniec , Romann Matthew Weber , Christopher Richard Schroers
- Applicant: LUCASFILM ENTERTAINMENT COMPANY LTD. LLC , DISNEY ENTERPRISES, INC
- Applicant Address: US CA San Francisco
- Assignee: LUCASFILM ENTERTAINMENT COMPANY LTD. LLC,DISNEY ENTERPRISES, INC
- Current Assignee: LUCASFILM ENTERTAINMENT COMPANY LTD. LLC,DISNEY ENTERPRISES, INC
- Current Assignee Address: US CA San Francisco; US CA Burbank
- Main IPC: G06T9/00
- IPC: G06T9/00 ; G06V40/16 ; G06T11/00 ; G06T7/70

Abstract:
A computer-implemented method of changing a face within an output image or video frame that includes: receiving an input image that includes a face presenting a facial expression in a pose; processing the image with a neural network encoder to generate a latent space point that is an encoded representation of the image; decoding the latent space point to generate an initial output image in accordance with a desired facial identity but with the facial expression and pose of the face in the input image; identifying a feature of the facial expression in the initial output image to edit; applying an adjustment vector to a latent space point corresponding to the initial output image to generate an adjusted latent space point; and decoding the adjusted latent space point to generate an adjusted output image in accordance with the desired facial identity but with the facial expression and pose of the face in the input image altered in accordance with the adjustment vector
Information query