公开(公告)号：US08744994B2

公开(公告)日：2014-06-03

申请号：US12343021

申请日：2008-12-23

Applicant: Ying Chen ,  Bin He ,  Rui Wang

Inventor： Ying Chen ,  Bin He ,  Rui Wang

IPC: G06F7/00 ,  G06F17/30

CPC classification number: G06F17/30563 ,  G06F17/30592 ,  G06F17/30938

Abstract: A method and system are disclosed for use with an ETL (Extract, Transform, Load) process, comprising optimizing a filter expression to select a subset of data and evaluating the filter expression on the data after the extracting, before the loading, but not during the transforming of the ETL process. The method and system optimizes the filtering using a pipelined evaluation for single predicate filtering and an adaptive optimization for multiple predicate filtering. The adaptive optimization includes an initial phase and a dynamic phase.

Abstract translation: 公开了一种与ETL（提取，变换，加载）过程一起使用的方法和系统，包括优化滤波器表达式以选择数据子集，并且在加载之前对提取之后的数据评估滤波器表达式，但不在 ETL流程的转型。 该方法和系统使用用于单个谓词过滤的流水线评估和针对多个谓词过滤的自适应优化来优化过滤。 自适应优化包括初始阶段和动态阶段。

公开(公告)号：US08429116B2

公开(公告)日：2013-04-23

申请号：US12131078

申请日：2008-05-31

Applicant: Ying Chen ,  Bin He

Inventor： Ying Chen ,  Bin He

IPC: G06F17/30

CPC classification number: G06F17/30592

Abstract: A system and method for ensuring large and frequent updates to a data warehouse. The process leverages a set of temporary staging tables to track the updates. A set of intermediate steps are performed to accomplish bulk deletions of the outdated changed records, and perform modifications to the map tables for models such as snowflake. Finally, bulk load operations load the updates and insert them into the final dimension tables. The process ensures performance comparable to insertion-only schemes with at most only slight performance degradation. Furthermore, a modified process is applied on the newfact data warehouse dimension model. The process can be readily adapted to handle star schema and other hierarchical data warehouse models.

Abstract translation: 一种用于确保对数据仓库进行大量和频繁更新的系统和方法。 该过程利用一组临时暂存表来跟踪更新。 执行一组中间步骤以完成对过时的已更改记录的批量删除，并对诸如雪花之类的模型执行对映射表的修改。 最后，批量加载操作加载更新并将它们插入到最终维度表中。 该过程确保性能与只插入方案相当，最多只有轻微的性能下降。 此外，修改后的流程应用于新元数据仓库维度模型。 该过程可以容易地适应于处理星型模式和其他分层数据仓库模型。

公开(公告)号：US08326797B2

公开(公告)日：2012-12-04

申请号：US12616612

申请日：2009-11-11

Applicant: Bin He ,  Ying Chen

Inventor： Bin He ,  Ying Chen

IPC: G06F17/30

CPC classification number: G06F17/30592 ,  G06F17/30439

Abstract: A method and system for coping with slice and dice operations in data warehouses is disclosed. An external approach may be utilized, creating queries using structured query language on a computer. An algorithm may be used to rewrite the queries. The resulting predicates may be joined to dimension tables corresponding to fact tables. An internal approach may be utilized, using aggregation functions with early aggregation for creating the queries. The results of the slice and dice operations may be outputted to a user on a computer monitor.

Abstract translation: 公开了一种用于处理数据仓库中的切片和骰子操作的方法和系统。 可以使用外部方法，在计算机上使用结构化查询语言创建查询。 可以使用算法来重写查询。 所得到的谓词可以连接到对应于事实表的维度表。 可以使用内部方法，使用具有早期聚合的聚合功能来创建查询。 切片和骰子操作的结果可以在计算机监视器上输出给用户。

公开(公告)号：US20090187602A1

公开(公告)日：2009-07-23

申请号：US12131078

申请日：2008-05-31

Applicant: Ying Chen ,  Bin He

Inventor： Ying Chen ,  Bin He

IPC: G06F17/30

CPC classification number: G06F17/30592

Abstract: A system and method for ensuring large and frequent updates to a data warehouse. The process leverages a set of temporary staging tables to track the updates. A set of intermediate steps are performed to accomplish bulk deletions of the outdated changed records, and perform modifications to the map tables for models such as snowflake. Finally, bulk load operations load the updates and insert them into the final dimension tables. The process ensures performance comparable to insertion-only schemes with at most only slight performance degradation. Furthermore, a modified process is applied on the newfact data warehouse dimension model. The process can be readily adapted to handle star schema and other hierarchical data warehouse models.

Abstract translation: 一种用于确保对数据仓库进行大量和频繁更新的系统和方法。 该过程利用一组临时暂存表来跟踪更新。 执行一组中间步骤以完成对过时的已更改记录的批量删除，并对诸如雪花之类的模型执行对映射表的修改。 最后，批量加载操作加载更新并将它们插入到最终维度表中。 该过程确保性能与只插入方案相当，最多只有轻微的性能下降。 此外，修改后的流程应用于新元数据仓库维度模型。 该过程可以容易地适应于处理星型模式和其他分层数据仓库模型。

公开(公告)号：US09646071B2

公开(公告)日：2017-05-09

申请号：US13607320

申请日：2012-09-07

Applicant: Bin He ,  Ying Chen

Inventor： Bin He ,  Ying Chen

IPC: G06F17/30

CPC classification number: G06F17/30592 ,  G06F17/30439

Abstract: A method and system for coping with slice and dice operations in data warehouses is disclosed. An external approach may be utilized, creating queries using structured query language on a computer. An algorithm may be used to rewrite the queries. The resulting predicates may be joined to dimension tables corresponding to fact tables. An internal approach may be utilized, using aggregation functions with early aggregation for creating the queries. The results of the slice and dice operations may be outputted to a user on a computer monitor.

公开(公告)号：US09348884B2

公开(公告)日：2016-05-24

申请号：US12128264

申请日：2008-05-28

Applicant: Ying Chen ,  Bin He ,  Ana Lelescu

Inventor： Ying Chen ,  Bin He ,  Ana Lelescu

IPC: G06F17/30

CPC classification number: G06F17/30592 ,  G06F17/30563 ,  G06F17/30569

Abstract: Techniques for reducing a number of computations in a data storage process are provided. One or more computational elements are identified in the data storage process. An ordered structure of one or more nodes is generated using the one or more computational elements. Each of the one or more nodes represents one or more computational elements. Further, a weight is assigned to each of the one or more nodes. An ordered structure of one or more reusable nodes is generated by deleting one or more nodes in accordance with the assigned weights. The ordered structure of one or more reusable nodes is utilized to reduce the number of computations in the data storage process. The data storage process converts data from a first format into a second format, and stores the data in the second format on a computer readable medium for data analysis purposes.

Abstract translation: 提供了用于减少数据存储过程中的多个计算的技术。 在数据存储过程中识别一个或多个计算元件。 使用一个或多个计算元件生成一个或多个节点的有序结构。 一个或多个节点中的每一个表示一个或多个计算元件。 此外，对一个或多个节点中的每一个分配权重。 通过根据分配的权重删除一个或多个节点来生成一个或多个可重用节点的有序结构。 利用一个或多个可重用节点的有序结构来减少数据存储过程中的计算次数。 数据存储过程将数据从第一格式转换成第二格式，并将数据以第二格式存储在计算机可读介质上用于数据分析目的。

公开(公告)号：US08056054B2

公开(公告)日：2011-11-08

申请号：US11759851

申请日：2007-06-07

Applicant: Ying Chen ,  Bin He ,  Ana Lelescu ,  James J. Rhodes ,  Rui Wang

Inventor： Ying Chen ,  Bin He ,  Ana Lelescu ,  James J. Rhodes ,  Rui Wang

IPC: G06F9/44

CPC classification number: G06F17/30592 ,  G06F11/1471 ,  G06F2201/80 ,  G06Q30/02

Abstract: A method for use with an information (or data) warehouse comprises managing the information warehouse with instructions in a declarative language. The instructions specify information warehouse-level tasks to be done without specifying certain details of how the tasks are to be implemented, for example, using databases and text indexers. The details are hidden from the user and include, for example, in an information warehouse having a FACT table that joins two or more dimension tables, details of database level operations when structured data are being handled, including database command line utilities, database drivers, and structured query language (SQL) statements; and details of text-indexing engines when unstructured data are being handled. The information warehouse is managed in a dynamic way in which different tasks—such as data loading tasks and information warehouse construction tasks—may be interleaved (i.e., there is no particular order in which the different tasks must be completed).

Abstract translation: 一种与信息（或数据）仓库一起使用的方法包括以声明性语言的指令管理信息仓库。 该指令指定要完成的信息仓库级任务，而不指定如何实现任务的某些细节，例如使用数据库和文本索引器。 详细信息是从用户隐藏的，并且包括例如在具有连接两个或多个维度表的FACT表的信息仓库中，当处理结构化数据时的数据库级别操作的细节，包括数据库命令行实用程序，数据库驱动程序， 和结构化查询语言（SQL）语句; 以及正在处理非结构化数据时文本索引引擎的细节。 信息仓库以动态的方式进行管理，其中不同的任务（例如数据加载任务和信息仓库构建任务）可以被交错（即，不存在必须完成不同任务的特定顺序）。

公开(公告)号：US20100161576A1

公开(公告)日：2010-06-24

申请号：US12343021

申请日：2008-12-23

Applicant: Ying Chen ,  Bin He ,  Rui Wang

Inventor： Ying Chen ,  Bin He ,  Rui Wang

IPC: G06F17/30

CPC classification number: G06F17/30563 ,  G06F17/30592 ,  G06F17/30938

Abstract: A method and system are disclosed for use with an ETL (Extract, Transform, Load) process, comprising optimizing a filter expression to select a subset of data and evaluating the filter expression on the data after the extracting, before the loading, but not during the transforming of the ETL process. The method and system optimizes the filtering using a pipelined evaluation for single predicate filtering and an adaptive optimization for multiple predicate filtering. The adaptive optimization includes an initial phase and a dynamic phase.

Abstract translation: 公开了一种与ETL（提取，变换，加载）过程一起使用的方法和系统，包括优化滤波器表达式以选择数据子集，并且在加载之前对提取之后的数据评估滤波器表达式，而不是在 ETL流程的转型。 该方法和系统使用用于单个谓词过滤的流水线评估和针对多个谓词过滤的自适应优化来优化过滤。 自适应优化包括初始阶段和动态阶段。

公开(公告)号：US20080307011A1

公开(公告)日：2008-12-11

申请号：US12134065

申请日：2008-06-05

Applicant: Ying Chen ,  Bin He ,  Ana Lelescu ,  James J. Rhodes ,  Rui Wang

Inventor： Ying Chen ,  Bin He ,  Ana Lelescu ,  James J. Rhodes ,  Rui Wang

IPC: G06F12/16 ,  G06F17/30

CPC classification number: G06F17/30309 ,  G06F11/1469 ,  G06F11/1471

Abstract: A method of data loading for large information warehouses includes performing checkpointing concurrently with data loading into an information warehouse, the checkpointing ensuring consistency among multiple tables; and recovering from a failure in the data loading using the checkpointing. A method is also disclosed for performing versioning concurrently with data loading into an information warehouse. The versioning method enables processing undo and redo operations of the data loading between a later version and a previous version. Data load failure recovery is performed without starting a data load from the beginning but rather from a latest checkpoint for data loading at an information warehouse level using a checkpoint process characterized by a state transition diagram having a multiplicity of states; and tracking state transitions among the states using a system state table.

公开(公告)号：US20080306987A1

公开(公告)日：2008-12-11

申请号：US12134011

申请日：2008-06-05

Applicant: Ying Chen ,  Bin He ,  Ana Lelescu ,  James J. Rhodes ,  Rui Wang

Inventor： Ying Chen ,  Bin He ,  Ana Lelescu ,  James J. Rhodes ,  Rui Wang

IPC: G06F12/16 ,  G06F7/00 ,  G06F17/30

CPC classification number: G06F17/30592 ,  G06F11/1471 ,  G06F2201/80 ,  G06Q30/02

Abstract: A method for use with an information (or data) warehouse comprises managing the information warehouse with instructions in a declarative language. The instructions specify information warehouse-level tasks to be done without specifying certain details of how the tasks are to be implemented, for example, using databases and text indexers. The details are hidden from the user and include, for example, in an information warehouse having a FACT table that joins two or more dimension tables, details of database level operations when structured data are being handled, including database command line utilities, database drivers, and structured query language (SQL) statements; and details of text-indexing engines when unstructured data are being handled. The information warehouse is managed in a dynamic way in which different tasks—such as data loading tasks and information warehouse construction tasks—may be interleaved (i.e., there is no particular order in which the different tasks must be completed).