公开(公告)号：US20240153247A1

公开(公告)日：2024-05-09

申请号：US18053851

申请日：2022-11-09

Applicant: Lemon Inc.

Inventor： Yifan Zhang ,  Daquan Zhou ,  Kai Wang ,  Jiashi Feng

IPC: G06V10/774 ,  G06V10/40 ,  G06V10/764 ,  G06V10/82

CPC classification number: G06V10/774 ,  G06V10/40 ,  G06V10/764 ,  G06V10/82 ,  G06V20/52

Abstract: Automatic data generation includes extracting latent features from an input image, adding a perturbation to the latent features, applying the perturbed latent features to a pre-trained generative model, and training an image generator with images output from the generative model.

公开(公告)号：US20240169479A1

公开(公告)日：2024-05-23

申请号：US18056444

申请日：2022-11-17

Applicant: Lemon Inc.

Inventor： Wei Min Wang ,  Daquan Zhou ,  Jiashi Feng

IPC: G06T3/40

CPC classification number: G06T3/4007 ,  G06T3/4053

Abstract: The present disclosure provides systems and methods for video generation using latent diffusion machine learning models. Given a text input, video data relevant to the text input can be generated using a latent diffusion model. The process includes generating a predetermined number of key frames using text-to-image generation tasks performed within a latent space via a variational auto-encoder, enabling faster training and sampling times compared to pixel space-based diffusion models. The process further includes utilizing two-dimensional convolutions and associated adaptors to learn features for a given frame. Temporal information for the frames can be learned via a directed temporal attention module used to capture the relation among frames and to generate a temporally meaningful sequence of frames. Additional frames can be generated via a frame interpolation process for inserting one or more transition frames between two generated frames. The process can also include a super-resolution process for upsampling the frames.

公开(公告)号：US20240144544A1

公开(公告)日：2024-05-02

申请号：US18050349

申请日：2022-10-27

Applicant: Lemon Inc.

Inventor： Jun Hao Liew ,  Hanshu Yan ,  Daquan Zhou ,  Jiashi Feng

IPC: G06T11/00 ,  G06F40/40 ,  G06T5/00 ,  G06T5/20

CPC classification number: G06T11/00 ,  G06F40/40 ,  G06T5/002 ,  G06T5/20 ,  G06T2207/20084

Abstract: Generating an object using a diffusion model includes obtaining a first input and a second input, and synthesizing an output object from the first input and the second input. The synthesizing of the output object includes generating a layout of the output object from the first input, injecting the second input as a content conditioner to the layout of the output object, and de-noising the layout of the output object injected with the content conditioner to generate a content of the output object.

公开(公告)号：US20220398450A1

公开(公告)日：2022-12-15

申请号：US17348246

申请日：2021-06-15

Applicant: Lemon Inc.

Inventor： Xiaojie JIN ,  Daquan Zhou ,  Xiaochen Lian ,  Linjie Yang ,  Jiashi Feng

IPC: G06N3/08 ,  G06N3/12

Abstract: A super-network comprising a plurality of layers may be generated. Each layer may comprise cells with different structures. A predetermined number of cells from each layer may be selected. A plurality of cells may be generated based on selected cells using a local mutation model, wherein the local mutation model comprises a mutation window for removing redundant edges from each selected cell. Performance of the plurality of cells may be evaluated using a differentiable fitness scoring function. The operations of the generating a plurality of cells using the local mutation model, the evaluating performance of the plurality of cells using the differentiable fitness scoring function and the selecting the subset of cells based on the evaluation results may be iteratively performed until the super-network converges. A search space for each layer may be generated based on a predetermined top number of cells with largest fitness scores after the super-network converges.