-
公开(公告)号:US11841845B2
公开(公告)日:2023-12-12
申请号:US17462938
申请日:2021-08-31
Applicant: LEMON INC.
Inventor: Jianjun Chen , Yonghua Ding , Ye Liu , Fangshi Li , Lixun Cao , Yang Liu , Li Zhang , Mingyi Zhang , Xiangrui Meng , Junda Zhao , Lei Zhang , Rui Shi
CPC classification number: G06F16/2365 , G06F16/2358 , G06F16/278 , G06F16/283
Abstract: The present disclosure describes techniques of providing data consistency for hybrid transactional and analytical processing. Logical logs and log serial numbers (LSNs) associated with the logical logs may be generated based on data captured by a first processing engine. The logical logs and the LSNs may be propagated to a storage subsystem configured to be in communication with the first processing engine and a second processing engine. The LSNs and information indicative of LSN schema versions may be stored and distributed by a metadata service. The first processing engine, the second processing engine, the storage subsystem and the metadata service are modularized, and support a LSN mechanism for maintaining data consistency.
-
公开(公告)号:US20230063730A1
公开(公告)日:2023-03-02
申请号:US17462853
申请日:2021-08-31
Applicant: LEMON INC.
Inventor: Jianjun Chen , Yonghua Ding , Ye Liu , Fangshi Li , Li Zhang , Mingyi Zhang , Kui Wei , Wei Ding , Kai Wu , Jason Yang Sun
Abstract: The present disclosure describes storage techniques for hybrid transactional and analytical processing. Data captured by a first processing engine may be received. The first processing engine may be configured to perform online transactional processing). Multiple replicas of logical logs generated based on the data may be distributed to a Delta Store by applying a quorum protocol on the multiple replicas. Data in the Delta Store are stored in a row format and are visible to a query for online analytical processing performed by a second processing engine. Data may be flushed from the Delta Store to a Base Store based on one or more predetermined rules. Data in the Base Store are stored in a columnar format and may be accessible by the second processing engine.
-
公开(公告)号:US20230062198A1
公开(公告)日:2023-03-02
申请号:US17462938
申请日:2021-08-31
Applicant: LEMON INC.
Inventor: Jianjun Chen , Yonghua Ding , Ye Liu , Fangshi Li , Lixun Cao , Yang Liu , Li Zhang , Mingyi Zhang , Xiangrui Meng , Junda Zhao , Lei Zhang , Rui Shi
Abstract: The present disclosure describes techniques of providing data consistency for hybrid transactional and analytical processing. Logical logs and log serial numbers (LSNs) associated with the logical logs may be generated based on data captured by a first processing engine. The logical logs and the LSNs may be propagated to a storage subsystem configured to be in communication with the first processing engine and a second processing engine. The LSNs and information indicative of LSN schema versions may be stored and distributed by a metadata service. The first processing engine, the second processing engine, the storage subsystem and the metadata service are modularized, and support a LSN mechanism for maintaining data consistency.
-
公开(公告)号:US12147432B2
公开(公告)日:2024-11-19
申请号:US17462998
申请日:2021-08-31
Applicant: LEMON INC.
Inventor: Jianjun Chen , Yonghua Ding , Ye Liu , Fangshi Li , Lixun Cao , Yang Liu , Li Zhang , Mingyi Zhang , Lei Zhang , Rui Shi
IPC: G06F16/2457 , G06F16/178 , G06F16/242 , G06F16/25 , G06F16/27
Abstract: The present disclosure describes hybrid transactional and analytical processing (HTAP) techniques. A HTAP system comprises a first processing engine configured to perform online transactional processing, a second processing engine configured to perform online analytical processing, and a storage in communication with the first processing engine and the second processing engine. The first processing engine, the second processing engine, and the storage may be modularized and configured to be decoupled from each other. The system may be configured to capture data by the first processing engine in real time, organize the data in a first format in a first part of the storage for use by the first processing engine, propagate the data to a second part of the storage subsystem, and organize the data in a second format in the second part of the storage for use by the second processing engine.
-
公开(公告)号:US11789936B2
公开(公告)日:2023-10-17
申请号:US17462853
申请日:2021-08-31
Applicant: LEMON INC.
Inventor: Jianjun Chen , Yonghua Ding , Ye Liu , Fangshi Li , Li Zhang , Mingyi Zhang , Kui Wei , Wei Ding , Kai Wu , Jason Yang Sun
CPC classification number: G06F16/2379 , G06F16/28
Abstract: The present disclosure describes storage techniques for hybrid transactional and analytical processing. Data captured by a first processing engine may be received. The first processing engine may be configured to perform online transactional processing). Multiple replicas of logical logs generated based on the data may be distributed to a Delta Store by applying a quorum protocol on the multiple replicas. Data in the Delta Store are stored in a row format and are visible to a query for online analytical processing performed by a second processing engine. Data may be flushed from the Delta Store to a Base Store based on one or more predetermined rules. Data in the Base Store are stored in a columnar format and may be accessible by the second processing engine.
-
6.
公开(公告)号:US20230289616A1
公开(公告)日:2023-09-14
申请号:US18319636
申请日:2023-05-18
Applicant: Lemon Inc.
Inventor: Tongping Liu , Wei Xu , Jianjun Chen
IPC: G06N3/098
CPC classification number: G06N3/098
Abstract: System and method of training a machine learning model on a plurality of devices in parallel are provided. The method includes performing a model profiling execution before a model normal execution, allocating tensors of the model into a plurality of chunks based on profiling results from the model profiling execution, and performing the model normal execution on the plurality of devices in parallel to train or fine-tune the model.
-
公开(公告)号:US20240403292A1
公开(公告)日:2024-12-05
申请号:US18798502
申请日:2024-08-08
Applicant: Lemon Inc. , Beijing Zitiao Network Technology Co., Ltd.
Inventor: Wei Ding , Li Zhang , Yuanjin Lin , Kui Wei , Yuxiang Chen , Jianjun Chen
IPC: G06F16/2453 , G06F11/30 , G06F11/34
Abstract: A hybrid transactional/analytical processing database includes an OLTP engine, OLAP engine, and shared storage including a mass storage device holding data of multiple tenants. The shared storage includes processing circuitry configured to implement a query executor configured to continuously monitor actual memory usage of the shared storage while the shared storage is responding to one or more queries from the OLTP engine or OLAP engine. Upon determining the actual memory usage of an operator evaluating the one or more of the queries from the OLTP engine or OLAP engine at the shared storage exceeds the preventative memory usage threshold but does not exceed the system memory usage threshold, the query executor is further configured to spill one or more fragments of query memory corresponding to the one or more queries to the mass storage device, without terminating the one or more queries.
-
公开(公告)号:US20230066540A1
公开(公告)日:2023-03-02
申请号:US17462998
申请日:2021-08-31
Applicant: LEMON INC.
Inventor: Jianjun Chen , Yonghua Ding , Ye Liu , Fangshi Li , Lixun Cao , Yang Liu , Li Zhang , Mingyi Zhang , Lei Zhang , Rui Shi
IPC: G06F16/2457 , G06F16/25 , G06F16/242 , G06F16/27
Abstract: The present disclosure describes hybrid transactional and analytical processing (HTAP) techniques. A HTAP system comprises a first processing engine configured to perform online transactional processing, a second processing engine configured to perform online analytical processing, and a storage in communication with the first processing engine and the second processing engine. The first processing engine, the second processing engine, and the storage may be modularized and configured to be decoupled from each other. The system may be configured to capture data by the first processing engine in real time, organize the data in a first format in a first part of the storage for use by the first processing engine, propagate the data to a second part of the storage subsystem, and organize the data in a second format in the second part of the storage for use by the second processing engine.
-
-
-
-
-
-
-