公开(公告)号：US11803996B2

公开(公告)日：2023-10-31

申请号：US17390440

申请日：2021-07-30

Applicant: Lemon Inc.

Inventor： Wanchun Ma ,  Shuo Cheng ,  Chao Wang ,  Michael Leong Hou Tay ,  Linjie Luo

IPC: G06T13/40 ,  G06N3/08 ,  G06V40/16

CPC classification number: G06T13/40 ,  G06N3/08 ,  G06V40/162 ,  G06V40/171 ,  G06V40/176

Abstract: Techniques for face tracking comprise receiving landmark data associated with a plurality of images indicative of at least one facial part. Representative images corresponding to the plurality of images may be generated based on the landmark data. Each representative image may depict a plurality of segments, and each segment may correspond to a region of the at least one facial part. The plurality of images and corresponding representative images may be input into a neural network to train the neural network to predict a feature associated with a subsequently received image comprising a face. An animation associated with a facial expression may be controlled based on output from the trained neural network.

公开(公告)号：US20230046286A1

公开(公告)日：2023-02-16

申请号：US17402344

申请日：2021-08-13

Applicant: Lemon Inc.

Inventor： Michael Leong Hou Tay ,  Wanchun Ma ,  Shuo Cheng ,  Chao Wang ,  Linjie Luo

IPC: G06K9/00 ,  G06K9/62 ,  G06K9/32 ,  G06T13/80 ,  G06T13/40 ,  G06T7/246

Abstract: The present disclosure describes techniques for facial expression recognition. A first loss function may be determined based on a first set of feature vectors associated with a first set of images depicting facial expressions and a first set of labels indicative of the facial expressions. A second loss function may be determined based on a second set of feature vectors associated with a second set of images depicting asymmetric facial expressions and a second set of labels indicative of the asymmetric facial expressions. The first loss function and the second loss function may be used to determine a maximum loss function. The maximum loss function may be applied during training of a model. The trained model may be configured to predict at least one asymmetric facial expression in a subsequently received image.

公开(公告)号：US12243292B2

公开(公告)日：2025-03-04

申请号：US17929449

申请日：2022-09-02

Applicant: Lemon Inc.

Inventor： Shuo Cheng ,  Wanchun Ma ,  Linjie Luo

IPC: G06K9/62 ,  G06N3/0455 ,  G06N3/09 ,  G06V10/44 ,  G06V10/764 ,  G06V10/766 ,  G06V10/774 ,  G06V10/776 ,  G06V10/778 ,  G06V10/82 ,  G06V10/96 ,  G06V40/16

Abstract: Systems and methods for multi-task joint training of a neural network including an encoder module and a multi-headed attention mechanism are provided. In one aspect, the system includes a processor configured to receive input data including a first set of labels and a second set of labels. Using the encoder module, features are extracted from the input data. Using a multi-headed attention mechanism, training loss metrics are computed. A first training loss metric is computed using the extracted features and the first set of labels, and a second training loss metric is computed using the extracted features and the second set of labels. A first mask is applied to filter the first training loss metric, and a second mask is applied to filter the second training loss metric. A final training loss metric is computed based on the filtered first and second training loss metrics.

公开(公告)号：US12112573B2

公开(公告)日：2024-10-08

申请号：US17402344

申请日：2021-08-13

Applicant: Lemon Inc.

Inventor： Michael Leong Hou Tay ,  Wanchun Ma ,  Shuo Cheng ,  Chao Wang ,  Linjie Luo

IPC: G06K9/00 ,  G06F18/21 ,  G06T7/246 ,  G06T13/40 ,  G06T13/80 ,  G06V10/24 ,  G06V40/16

CPC classification number: G06V40/176 ,  G06F18/2193 ,  G06T7/251 ,  G06T13/40 ,  G06T13/80 ,  G06V10/242 ,  G06V40/171 ,  G06T2207/20084 ,  G06T2207/30201

Abstract: The present disclosure describes techniques for facial expression recognition. A first loss function may be determined based on a first set of feature vectors associated with a first set of images depicting facial expressions and a first set of labels indicative of the facial expressions. A second loss function may be determined based on a second set of feature vectors associated with a second set of images depicting asymmetric facial expressions and a second set of labels indicative of the asymmetric facial expressions. The first loss function and the second loss function may be used to determine a maximum loss function. The maximum loss function may be applied during training of a model. The trained model may be configured to predict at least one asymmetric facial expression in a subsequently received image.

公开(公告)号：US20240265621A1

公开(公告)日：2024-08-08

申请号：US18165794

申请日：2023-02-07

Applicant: Lemon Inc.

Inventor： Hongyi Xu ,  Guoxian Song ,  Zihang Jiang ,  Jianfeng Zhang ,  Yichun Shi ,  Jing Liu ,  Wanchun Ma ,  Jiashi Feng ,  Linjie Luo

IPC: G06T15/08 ,  G06T3/40 ,  G06V40/16

CPC classification number: G06T15/08 ,  G06T3/4046 ,  G06T3/4053 ,  G06V40/176

Abstract: Technologies are described and recited herein for producing controllable synthesized images include a geometry guided 3D GAN framework for high-quality 3D head synthesis with full control on camera poses, facial expressions, head shape, articulated neck and jaw poses; and a semantic SDF (signed distance function) formulation that defines volumetric correspondence from observation space to canonical space, allowing full disentanglement of control parameters in 3D GAN training.

公开(公告)号：US20240078792A1

公开(公告)日：2024-03-07

申请号：US17929449

申请日：2022-09-02

Applicant: Lemon Inc.

Inventor： Shuo Cheng ,  Wanchun Ma ,  Linjie Luo

IPC: G06V10/774 ,  G06V10/764 ,  G06V10/776 ,  G06V10/82 ,  G06V10/96 ,  G06V40/16

CPC classification number: G06V10/774 ,  G06V10/764 ,  G06V10/776 ,  G06V10/82 ,  G06V10/96 ,  G06V40/171 ,  G06V40/174

Abstract: Systems and methods for multi-task joint training of a neural network including an encoder module and a multi-headed attention mechanism are provided. In one aspect, the system includes a processor configured to receive input data including a first set of labels and a second set of labels. Using the encoder module, features are extracted from the input data. Using a multi-headed attention mechanism, training loss metrics are computed. A first training loss metric is computed using the extracted features and the first set of labels, and a second training loss metric is computed using the extracted features and the second set of labels. A first mask is applied to filter the first training loss metric, and a second mask is applied to filter the second training loss metric. A final training loss metric is computed based on the filtered first and second training loss metrics.

公开(公告)号：US11720994B2

公开(公告)日：2023-08-08

申请号：US17321384

申请日：2021-05-14

Applicant: Lemon Inc.

Inventor： Linjie Luo ,  Guoxian Song ,  Jing Liu ,  Wanchun Ma

IPC: G06T3/00 ,  G06T11/00 ,  G06T5/00 ,  G06N3/08 ,  G06F18/214 ,  G06N3/045

CPC classification number: G06T3/0012 ,  G06F18/214 ,  G06N3/045 ,  G06N3/08 ,  G06T3/0006 ,  G06T5/00 ,  G06T11/00 ,  G06T2207/20016 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30201

Abstract: Systems and method directed to an inversion-consistent transfer learning framework for generating portrait stylization using only limited exemplars. In examples, an input image is received and encoded using a variational autoencoder to generate a latent vector. The latent vector may be provided to a generative adversarial network (GAN) generator to generate a stylized image. In examples, the variational autoencoder is trained using a plurality of images while keeping the weights of a pre-trained GAN generator fixed, where the pre-trained GAN generator acts as a decoder for the encoder. In other examples, a multi-path attribute aware generator is trained using a plurality of exemplar images and learning transfer using the pre-trained GAN generator.