-
公开(公告)号:US12148095B2
公开(公告)日:2024-11-19
申请号:US17932640
申请日:2022-09-15
Applicant: Lemon Inc.
Inventor: Tiancheng Zhi , Shen Sang , Guoxian Song , Chunpong Lai , Jing Liu , Linjie Luo
Abstract: Systems and methods for rendering a translucent object are provided. In one aspect, the system includes a processor coupled to a storage medium that stores instructions, which, upon execution by the processor, cause the processor to receive at least one mesh representing at least one translucent object. For each pixel to be rendered, the processor performs a rasterization-based differentiable rendering of the pixel to be rendered using the at least one mesh and determines a plurality of values for the pixel to be rendered based on the rasterization-based differentiable rendering. The rasterization-based differentiable rendering can include performing a probabilistic rasterization process along with aggregation techniques to compute the plurality of values for the pixel to be rendered. The plurality of values includes a set of color channel values and an opacity channel value. Once values are determined for all pixels, an image can be rendered.
-
公开(公告)号:US12051168B2
公开(公告)日:2024-07-30
申请号:US17932645
申请日:2022-09-15
Applicant: Lemon Inc. , Beijing Zitiao Network Technology Co., Ltd.
Inventor: Hongyi Xu , Tao Hu , Linjie Luo
CPC classification number: G06T19/20 , G06T7/75 , G06T15/04 , G06T17/20 , G06T2207/20084 , G06T2210/16 , G06T2219/2004
Abstract: Systems and methods are provided that include a processor executing an avatar generation program to obtain driving view(s), calculate a skeletal pose of the user, and generate a coarse human mesh based on a template mesh and the skeletal pose of the user. The program further constructs a texture map based on the driving view(s) and the coarse human mesh, extracts a plurality of image features from the texture map, the image features being aligned to a UV map, and constructs a UV positional map based on the coarse human mesh. The program further extracts a plurality of pose features from the UV positional map, the pose features being aligned to the UV map, generates a plurality of pose-image features based on the UV map-aligned image features and UV map-aligned pose features, and renders an avatar based on the plurality of pose-image features.
-
公开(公告)号:US20230410267A1
公开(公告)日:2023-12-21
申请号:US17807527
申请日:2022-06-17
Applicant: Lemon Inc. , Beijing Zitiao Network Technology Co., Ltd.
Inventor: Guoxian Song , Jing Liu , Weihong Zeng , Jingna Sun , Xu Wang , Linjie Luo
CPC classification number: G06T5/50 , G06T3/4046 , G06V40/168 , G06T2207/20084 , G06T2207/20132 , G06T2207/20081 , G06T2207/20221 , G06T2207/30201
Abstract: Methods and systems for enlarging a stylized region of an image are disclosed that include receiving an input image, generating, using a first generative adversarial network (GAN) generator, a first stylized image, based on the input image, normalizing the input image, generating, using a second generative adversarial network (GAN) generator, a second stylized image, based on the normalized input image, blending the first stylized image and the second stylized image to obtain a third stylized image, and providing the third stylized image as an output.
-
公开(公告)号:US11803996B2
公开(公告)日:2023-10-31
申请号:US17390440
申请日:2021-07-30
Applicant: Lemon Inc.
Inventor: Wanchun Ma , Shuo Cheng , Chao Wang , Michael Leong Hou Tay , Linjie Luo
CPC classification number: G06T13/40 , G06N3/08 , G06V40/162 , G06V40/171 , G06V40/176
Abstract: Techniques for face tracking comprise receiving landmark data associated with a plurality of images indicative of at least one facial part. Representative images corresponding to the plurality of images may be generated based on the landmark data. Each representative image may depict a plurality of segments, and each segment may correspond to a region of the at least one facial part. The plurality of images and corresponding representative images may be input into a neural network to train the neural network to predict a feature associated with a subsequently received image comprising a face. An animation associated with a facial expression may be controlled based on output from the trained neural network.
-
公开(公告)号:US20230046286A1
公开(公告)日:2023-02-16
申请号:US17402344
申请日:2021-08-13
Applicant: Lemon Inc.
Inventor: Michael Leong Hou Tay , Wanchun Ma , Shuo Cheng , Chao Wang , Linjie Luo
Abstract: The present disclosure describes techniques for facial expression recognition. A first loss function may be determined based on a first set of feature vectors associated with a first set of images depicting facial expressions and a first set of labels indicative of the facial expressions. A second loss function may be determined based on a second set of feature vectors associated with a second set of images depicting asymmetric facial expressions and a second set of labels indicative of the asymmetric facial expressions. The first loss function and the second loss function may be used to determine a maximum loss function. The maximum loss function may be applied during training of a model. The trained model may be configured to predict at least one asymmetric facial expression in a subsequently received image.
-
公开(公告)号:US12243292B2
公开(公告)日:2025-03-04
申请号:US17929449
申请日:2022-09-02
Applicant: Lemon Inc.
Inventor: Shuo Cheng , Wanchun Ma , Linjie Luo
IPC: G06K9/62 , G06N3/0455 , G06N3/09 , G06V10/44 , G06V10/764 , G06V10/766 , G06V10/774 , G06V10/776 , G06V10/778 , G06V10/82 , G06V10/96 , G06V40/16
Abstract: Systems and methods for multi-task joint training of a neural network including an encoder module and a multi-headed attention mechanism are provided. In one aspect, the system includes a processor configured to receive input data including a first set of labels and a second set of labels. Using the encoder module, features are extracted from the input data. Using a multi-headed attention mechanism, training loss metrics are computed. A first training loss metric is computed using the extracted features and the first set of labels, and a second training loss metric is computed using the extracted features and the second set of labels. A first mask is applied to filter the first training loss metric, and a second mask is applied to filter the second training loss metric. A final training loss metric is computed based on the filtered first and second training loss metrics.
-
17.
公开(公告)号:US12217466B2
公开(公告)日:2025-02-04
申请号:US17519711
申请日:2021-11-05
Applicant: Lemon Inc.
Inventor: Jing Liu , Chunpong Lai , Guoxian Song , Linjie Luo
Abstract: Systems and methods directed to controlling the similarity between stylized portraits and an original photo are described. In examples, an input image is received and encoded using a variational autoencoder to generate a latent vector. The latent vector may be blended with latent vectors that best represent a face in the original user portrait image. The resulting blended latent vector may be provided to a generative adversarial network (GAN) generator to generate a controlled stylized image. In examples, one or more layers of the stylized GAN generator may be swapped with one or more layers of the original GAN generator. Accordingly, a user can interactively determine how much stylization vs. personalization should be included in a resulting stylized portrait.
-
公开(公告)号:US12112573B2
公开(公告)日:2024-10-08
申请号:US17402344
申请日:2021-08-13
Applicant: Lemon Inc.
Inventor: Michael Leong Hou Tay , Wanchun Ma , Shuo Cheng , Chao Wang , Linjie Luo
CPC classification number: G06V40/176 , G06F18/2193 , G06T7/251 , G06T13/40 , G06T13/80 , G06V10/242 , G06V40/171 , G06T2207/20084 , G06T2207/30201
Abstract: The present disclosure describes techniques for facial expression recognition. A first loss function may be determined based on a first set of feature vectors associated with a first set of images depicting facial expressions and a first set of labels indicative of the facial expressions. A second loss function may be determined based on a second set of feature vectors associated with a second set of images depicting asymmetric facial expressions and a second set of labels indicative of the asymmetric facial expressions. The first loss function and the second loss function may be used to determine a maximum loss function. The maximum loss function may be applied during training of a model. The trained model may be configured to predict at least one asymmetric facial expression in a subsequently received image.
-
公开(公告)号:US20240265621A1
公开(公告)日:2024-08-08
申请号:US18165794
申请日:2023-02-07
Applicant: Lemon Inc.
Inventor: Hongyi Xu , Guoxian Song , Zihang Jiang , Jianfeng Zhang , Yichun Shi , Jing Liu , Wanchun Ma , Jiashi Feng , Linjie Luo
CPC classification number: G06T15/08 , G06T3/4046 , G06T3/4053 , G06V40/176
Abstract: Technologies are described and recited herein for producing controllable synthesized images include a geometry guided 3D GAN framework for high-quality 3D head synthesis with full control on camera poses, facial expressions, head shape, articulated neck and jaw poses; and a semantic SDF (signed distance function) formulation that defines volumetric correspondence from observation space to canonical space, allowing full disentanglement of control parameters in 3D GAN training.
-
公开(公告)号:US20240135627A1
公开(公告)日:2024-04-25
申请号:US18046077
申请日:2022-10-12
Applicant: Lemon INc.
Inventor: Guoxian SONG , Shen Sang , Tiancheng Zhi , Jing Liu , Linjie Luo
CPC classification number: G06T15/02 , G06T7/11 , G06T2207/20081 , G06T2207/20084 , G06T2207/30201
Abstract: A method of generating a style image is described. The method includes receiving an input image of a subject. The method further includes encoding the input image using a first encoder of a generative adversarial network (GAN) to obtain a first latent code. The method further includes decoding the first latent code using a first decoder of the GAN to obtain a normalized style image of the subject, wherein the GAN is trained using a loss function according to semantic regions of the input image and the normalized style image.
-
-
-
-
-
-
-
-
-