公开(公告)号：US20250056084A1

公开(公告)日：2025-02-13

申请号：US18711497

申请日：2022-11-18

Applicant: LEMON INC.

Inventor： Xiaojie JIN ,  Weibo GONG ,  Quanwei HUANG ,  Xiaohui SHEN

IPC: H04N21/431 ,  G06V10/42 ,  G06V10/44 ,  H04N21/81 ,  H04N21/845

Abstract: The embodiments of the present disclosure provide a video generation method, an apparatus, an electronic device, a storage medium, a computer program product and a computer program, the method including: obtaining a plurality of video segments; determining feature information corresponding to the plurality of video segments; according to the feature information and a plurality of pre-stored rendering effects, determining an effect combination to be added; the rendering effects being animation, special effects or a transition; and generating a target video according to the plurality of video segments and the effect combination to be added.

公开(公告)号：US20240420458A1

公开(公告)日：2024-12-19

申请号：US18744418

申请日：2024-06-14

Applicant: Lemon Inc. ,  BEIJING ZITIAO NETWORK TECHNOLOGY CO., LTD. ,  INSTITUTE OF AUTOMATION CHINESE ACADEMY OF SCIENCES

Inventor： Xiaojie JIN ,  Sihan CHEN ,  Jiashi FENG ,  Xingjian HE ,  Handong LI ,  Jing LIU

IPC: G06V10/80 ,  G06V10/44 ,  G06V10/74

Abstract: The disclosure provides a cross-modal data processing method and apparatus, a device, a storage medium, and a program product. The method comprises: obtaining first modal data to be processed; obtaining a first modal data feature by performing feature extraction based on the first modal data; and obtaining second modal data based on the first modal data feature and a cross-modal processing model, the first modal data and the second modal data having different modalities, wherein the cross-modal processing model needs to be pre-trained based on a concatenated training sample, and the concatenated training sample comprises a concatenated image sample and a corresponding concatenated text sample.

公开(公告)号：US20250071390A1

公开(公告)日：2025-02-27

申请号：US18723804

申请日：2022-12-15

Applicant: Lemon Inc.

Inventor： Weibo GONG ,  Xiaojie JIN ,  Xiaohui SHEN

IPC: H04N21/81 ,  G10H1/00 ,  H04N21/439 ,  H04N21/44

Abstract: The present disclosure provides a video generation method based on music beats, a video generation apparatus based on music beats, an electronic device and a computer-readable storage medium. The method includes: acquiring a plurality of video objects and audio information respectively; determining a plurality of initial music beats in the audio information and characteristic information of each initial music beat, in which the characteristic information at least includes a sound intensity of each initial music beat and time of each initial music beat in the audio information; according to the characteristic information, selecting a target music beat from the plurality of initial music beats; and generating a target video according to the target music beat and the plurality of video objects.

公开(公告)号：US20240241907A1

公开(公告)日：2024-07-18

申请号：US18570310

申请日：2022-05-10

Applicant: LEMON INC.

Inventor： Ding LIU ,  Xiaojie JIN ,  Yan WANG ,  Weibo GONG

IPC: G06F16/68 ,  G06F16/55 ,  G06F16/65 ,  G06V10/764

CPC classification number: G06F16/68 ,  G06F16/55 ,  G06F16/65 ,  G06V10/764

Abstract: Embodiments of the disclosure provide a method, device, storage medium and program product for music screening. The method includes: obtaining at least one image and at least one piece of to-be-selected music; determining an analysis result of the at least one image corresponding to an image classification tag based on N predetermined image classification tags, N being an integer greater than or equal to 1; determining attribute information for each piece of to-be-selected music based on the at least one image and the at least one piece of to-be-selected music; determining target music that matches the at least one image among the at least one piece of to-be-selected music based on the analysis result and the attribute information of each piece of to-be-selected music.

公开(公告)号：US20250097545A1

公开(公告)日：2025-03-20

申请号：US18711530

申请日：2022-11-18

Applicant: Lemon Inc.

Inventor： Weibo GONG ,  Xiaojie JIN ,  Ding LIU ,  Xiaohui SHEN

IPC: H04N21/81 ,  G06V10/42 ,  G06V10/44

Abstract: The embodiments of the present disclosure provide a video generation method, an apparatus, a device, and a storage medium, the video generation method including: obtaining a plurality of images and music matched to the plurality of images; determining first feature information for the plurality of images and second feature information for the music; according to the first feature information, the second feature information and a plurality of pre-stored rendering effects, determining a target rendering effect combination; the rendering effects being animation, special effects or a transition; and generating a video according to the plurality of images, the music and the target rendering effect combination.

公开(公告)号：US20240290016A1

公开(公告)日：2024-08-29

申请号：US18570533

申请日：2022-05-09

Applicant: Lemon Inc.

Inventor： Xiaojie JIN ,  Yan WANG

IPC: G06T11/60 ,  G06V10/44 ,  G06V10/46 ,  G06V40/16

CPC classification number: G06T11/60 ,  G06V10/44 ,  G06V10/462 ,  G06V40/168

Abstract: The present disclosure relates to an image processing method, an apparatus, and a readable storage medium. The image processing method may obtain a multimedia resource with content continuity by: performing feature analysis on at least one image material to acquire a content feature set, where each content feature included in the content feature set is used for representing a content expressed by the image material in a corresponding particular dimension; next, acquiring an editing strategy of the at least one image material based on the content feature set and in accordance with a mapping relationship between different content features in the particular dimension and different operation modes of different editing operation types; and synthesizing the at least one image material according to each target editing operation mode included in the editing strategy.

公开(公告)号：US20220398402A1

公开(公告)日：2022-12-15

申请号：US17348181

申请日：2021-06-15

Applicant: Lemon Inc.

Inventor： Xiaojie JIN ,  Yi-Wen Chen ,  Xiaohui Shen

IPC: G06K9/00 ,  G06K9/46 ,  G06K9/62

Abstract: The present disclosure describes techniques of detecting objects in a video. The techniques comprises extracting features from each frame of the video; generating a first attentive feature by applying a first attention model on at least some of features extracted from any particular frame among the plurality of frames, wherein the first attention model identifies correlations between a plurality of locations in the particular frame by computing relationships between any two locations among the plurality of locations; generating a second attentive feature by applying a second attention model on at least one pair of features at different levels selected from the features extracted from the particular frame, wherein the second attention model identifies a correlation between at least one pair of locations corresponding to the at least one pair of features; and generating a representation of an object included in the particular frame.

公开(公告)号：US20240395061A1

公开(公告)日：2024-11-28

申请号：US18671708

申请日：2024-05-22

Applicant: Lemon Inc. ,  Beijing Zitiao Network Technology Co., Ltd. ,  Institute of Automation Chinese Academy of Sciences

Inventor： Xiaojie JIN ,  Xingjian HE ,  Sihan CHEN ,  Fan MA ,  Zhicheng HUANG ,  Jing LIU ,  Jiashi FENG

IPC: G06V20/70 ,  G06V10/774 ,  G06V10/80 ,  G06V20/40

Abstract: The present disclosure provides a video processing method, apparatus, device, storage medium, and program product. The method includes: acquiring video data; obtaining, based on the video data, a temporal image feature with temporal information; determining, based on the temporal image feature, a target text feature in a set of text features that matches the temporal image feature; and obtaining, based on the target text feature, target text data corresponding to the video data.

公开(公告)号：US20240233350A1

公开(公告)日：2024-07-11

申请号：US18408967

申请日：2024-01-10

Applicant: Lemon Inc. ,  Beijing Zitiao Network Technology Co., Ltd.

Inventor： Xiaojie JIN ,  Fan MA ,  Jiashi FENG ,  Heng WANG ,  Jingjia HUANG

IPC: G06V10/80 ,  G06F40/284 ,  G06V10/774 ,  G06V20/40

CPC classification number: G06V10/806 ,  G06F40/284 ,  G06V10/774 ,  G06V20/46

Abstract: The embodiments of the disclosure provides a processing method, apparatus, electronic device and non-transitory computer-readable storage medium for multimodal data, wherein the method includes: obtaining data to be processed of an original modality; determining result data of a target modality corresponding to the data to be processed by processing the data to be processed with a target processing model; wherein the target processing model comprises a multimodal submodel, and the pre-training task of the multimodal submodel includes a task of locating local data that matches second modal data from first modal data; wherein when the first modal data belongs to the original modality, the second modal data belongs to the target modality; when the first modal data belongs to the target modality, the second modal data belongs to the original modality.

公开(公告)号：US20240233070A1

公开(公告)日：2024-07-11

申请号：US18406910

申请日：2024-01-08

Applicant: Lemon Inc. ,  Beijing Zitiao Network Technology Co., Ltd.

Inventor： Xiaojie JIN ,  Bowen ZHANG ,  Jiashi FENG

IPC: G06T3/40 ,  G06V10/77 ,  G06V10/82

CPC classification number: G06T3/40 ,  G06V10/7715 ,  G06V10/82

Abstract: Embodiments of the disclosure disclose a method, apparatus, electronic device and storage medium for multi-modal data processing, wherein the method includes: acquiring data of original modality; and processing the data of the original modality by a target processing model to determine data of target modality corresponding to the data of the original modality; wherein the target processing model comprises a multi-modal pre-trained sub-model and a multi-modal feature correction sub-model; a training process of the target processing model comprises training the multi-modal feature correction sub-model with parameters of the multi-modal pre-training sub-model fixed.