公开(公告)号：US08655859B2

公开(公告)日：2014-02-18

申请号：US12714929

申请日：2010-03-01

Applicant: Ying Chen ,  Bin He ,  Rui Wang

Inventor： Ying Chen ,  Bin He ,  Rui Wang

IPC: G06F7/00 ,  G06F17/00

CPC classification number: G06F17/30563 ,  G06F17/30359

Abstract: System and methods manage concurrent ETL processes accessing a database. Exemplary embodiments include a method for concurrency management for ETL processes in a database having database tables and communicatively coupled to a computer, the method including establishing a session lock for the database, determining that a current ETL process is accessing the database at a current time, associating a current expiration time with the session lock, the expiration time being stored in a lock table in the database, sending the session lock to the current ETL process and performing ETL-level locking for the current ETL process.

Abstract translation: 系统和方法管理访问数据库的并发ETL进程。 示例性实施例包括用于具有数据库表并且通信地耦合到计算机的数据库中的ETL进程的并发管理的方法，所述方法包括为数据库建立会话锁定，确定当前ETL进程正在当前时间访问数据库， 将当前到期时间与会话锁相关联，到期时间存储在数据库中的锁定表中，将会话锁定发送到当前ETL进程，并为当前ETL进程执行ETL级别锁定。

公开(公告)号：US08479158B2

公开(公告)日：2013-07-02

申请号：US12134011

申请日：2008-06-05

Applicant: Ying Chen ,  Bin He ,  Ana Lelescu ,  James J. Rhodes ,  Rui Wang

Inventor： Ying Chen ,  Bin He ,  Ana Lelescu ,  James J. Rhodes ,  Rui Wang

IPC: G06F9/44

CPC classification number: G06F17/30592 ,  G06F11/1471 ,  G06F2201/80 ,  G06Q30/02

Abstract: A method for use with an information (or data) warehouse comprises managing the information warehouse with instructions in a declarative language. The instructions specify information warehouse-level tasks to be done without specifying certain details of how the tasks are to be implemented, for example, using databases and text indexers. The details are hidden from the user and include, for example, in an information warehouse having a FACT table that joins two or more dimension tables, details of database level operations when structured data are being handled, including database command line utilities, database drivers, and structured query language (SQL) statements; and details of text-indexing engines when unstructured data are being handled. The information warehouse is managed in a dynamic way in which different tasks—such as data loading tasks and information warehouse construction tasks—may be interleaved (i.e., there is no particular order in which the different tasks must be completed).

Abstract translation: 一种与信息（或数据）仓库一起使用的方法包括以声明性语言的指令管理信息仓库。 该指令指定要完成的信息仓库级任务，而不指定如何实现任务的某些细节，例如使用数据库和文本索引器。 这些细节从用户隐藏起来，例如包括在具有连接两个或多个维度表的FACT表的信息仓库中，当处理结构化数据时的数据库级别操作的细节，包括数据库命令行实用程序，数据库驱动程序， 和结构化查询语言（SQL）语句; 以及正在处理非结构化数据时文本索引引擎的细节。 信息仓库以动态的方式进行管理，其中不同的任务（例如数据加载任务和信息仓库构建任务）可以被交错（即，不存在必须完成不同任务的特定顺序）。

公开(公告)号：US08078570B2

公开(公告)日：2011-12-13

申请号：US12434378

申请日：2009-05-01

Applicant: Ying Chen ,  Bin He ,  William Scott Spangler

Inventor： Ying Chen ,  Bin He ,  William Scott Spangler

IPC: G06F17/30

CPC classification number: G06F17/30592

Abstract: A method, system, and computer program product are disclosed. Exemplary embodiments of the method, system, and computer program product may include hardware, process steps, and computer program instructions for supporting versioning in a data warehouse. The data warehouse may include a data warehouse engine for creating a data warehouse including a fact table and temporary tables. Updated or new data records may be transferred into the data warehouse and bulk loaded into the temporary tables. The updated or new data records may be evaluated for attributes matching existing data records. A version number may be assigned to data records and data records may be marked as being the most current version. Updated and new data records may be bulk loaded from the temporary tables into the fact table when a version number or a version status is calculated.

Abstract translation: 公开了一种方法，系统和计算机程序产品。 方法，系统和计算机程序产品的示例性实施例可以包括用于支持数据仓库中的版本控制的硬件，处理步骤和计算机程序指令。 数据仓库可以包括用于创建包括事实表和临时表的数据仓库的数据仓库引擎。 更新的或新的数据记录可能会被传输到数据仓库中并批量加载到临时表中。 可以针对与现有数据记录匹配的属性来评估更新的或新的数据记录。 可以将版本号分配给数据记录，并且可以将数据记录标记为最新版本。 当计算版本号或版本状态时，更新的新数据记录可能会从临时表批量加载到事实表中。

公开(公告)号：US08055609B2

公开(公告)日：2011-11-08

申请号：US12017511

申请日：2008-01-22

Applicant: Ying Chen ,  Bin He

Inventor： Ying Chen ,  Bin He

IPC: G06F17/00

CPC classification number: G06F17/30592

Abstract: A system and method for ensuring large and frequent updates to a data warehouse. The process leverages a set of temporary staging tables to track the updates. A set of intermediate steps are performed to accomplish bulk deletions of the outdated changed records, and perform modifications to the map tables for models such as snowflake. Finally, bulk load operations load the updates and insert them into the final dimension tables. The process ensures performance comparable to insertion-only schemes with at most only slight performance degradation. Furthermore, a modified process is applied on the newfact data warehouse dimension model. The process can be readily adapted to handle star schema and other hierarchical data warehouse models.

Abstract translation: 一种用于确保对数据仓库进行大量和频繁更新的系统和方法。 该过程利用一组临时暂存表来跟踪更新。 执行一组中间步骤以完成对过时的已更改记录的批量删除，并对诸如雪花之类的模型执行对映射表的修改。 最后，批量加载操作加载更新并将它们插入到最终维度表中。 该过程确保性能与只插入方案相当，最多只有轻微的性能下降。 此外，修改后的流程应用于新元数据仓库维度模型。 该过程可以容易地适应于处理星型模式和其他分层数据仓库模型。

公开(公告)号：US07739547B2

公开(公告)日：2010-06-15

申请号：US11759856

申请日：2007-06-07

Applicant: Ying Chen ,  Bin He ,  Ana Lelescu ,  James J. Rhodes ,  Rui Wang

Inventor： Ying Chen ,  Bin He ,  Ana Lelescu ,  James J. Rhodes ,  Rui Wang

IPC: G06F11/00

CPC classification number: G06F17/30309 ,  G06F11/1469 ,  G06F11/1471

Abstract: A method of data loading for large information warehouses includes performing checkpointing concurrently with data loading into an information warehouse, the checkpointing ensuring consistency among multiple tables; and recovering from a failure in the data loading using the checkpointing. A method is also disclosed for performing versioning concurrently with data loading into an information warehouse. The versioning method enables processing undo and redo operations of the data loading between a later version and a previous version. Data load failure recovery is performed without starting a data load from the beginning but rather from a latest checkpoint for data loading at an information warehouse level using a checkpoint process characterized by a state transition diagram having a multiplicity of states; and tracking state transitions among the states using a system state table.

Abstract translation: 大型信息仓库的数据加载方法包括：将数据加载到信息仓库中同时进行检查点检查，确保多个表格之间的一致性; 并使用检查点从数据加载失败中恢复。 还公开了一种与数据加载到信息仓库中同时进行版本控制的方法。 版本控制方法可以处理在更高版本和先前版本之间的数据加载的撤消和重做操作。 执行数据加载失败恢复，而不从一开始就开始数据加载，而是从最新检查点开始，使用特征在于具有多个状态的状态转换图的检查点进程在信息仓库级别进行数据加载。 并使用系统状态表来跟踪状态之间的状态转换。

公开(公告)号：US20090300038A1

公开(公告)日：2009-12-03

申请号：US12128264

申请日：2008-05-28

Applicant: Ying Chen ,  Bin He ,  Ana Lelescu

Inventor： Ying Chen ,  Bin He ,  Ana Lelescu

IPC: G06F17/30 ,  G06F17/14

CPC classification number: G06F17/30592 ,  G06F17/30563 ,  G06F17/30569

Abstract: Techniques for reducing a number of computations in a data storage process are provided. One or more computational elements are identified in the data storage process. An ordered structure of one or more nodes is generated using the one or more computational elements. Each of the one or more nodes represents one or more computational elements. Further, a weight is assigned to each of the one or more nodes. An ordered structure of one or more reusable nodes is generated by deleting one or more nodes in accordance with the assigned weights. The ordered structure of one or more reusable nodes is utilized to reduce the number of computations in the data storage process. The data storage process converts data from a first format into a second format, and stores the data in the second format on a computer readable medium for data analysis purposes.

Abstract translation: 提供了用于减少数据存储过程中的多个计算的技术。 在数据存储过程中识别一个或多个计算元件。 使用一个或多个计算元件生成一个或多个节点的有序结构。 一个或多个节点中的每一个表示一个或多个计算元件。 此外，对一个或多个节点中的每一个分配权重。 通过根据分配的权重删除一个或多个节点来生成一个或多个可重用节点的有序结构。 利用一个或多个可重用节点的有序结构来减少数据存储过程中的计算次数。 数据存储过程将数据从第一格式转换成第二格式，并将数据以第二格式存储在计算机可读介质上用于数据分析目的。

公开(公告)号：US08572124B2

公开(公告)日：2013-10-29

申请号：US12715997

申请日：2010-03-02

Applicant: Amit Behal ,  Ying Chen ,  Bin He

Inventor： Amit Behal ,  Ying Chen ,  Bin He

IPC: G06F7/00 ,  G06F17/30 ,  G06F3/048

CPC classification number: G06F17/30604 ,  G06F9/44 ,  G06F17/30

Abstract: Data may be modeled as an undirected graph. A set of entities and a set of attributes may be defined. A set of relationships may be defined to represent semantic associations with each association connecting at least two entities. Attributes may be associated with entities rather than with relationships. A hierarchical query language with a set of atomic operations on modeled data may be employed. The modeled data may be displayed on a display unit.

Abstract translation: 数据可以被建模为无向图。 可以定义一组实体和一组属性。 可以定义一组关系以表示与连接至少两个实体的每个关联的语义关联。 属性可能与实体相关联，而不是与关系关联。 可以采用具有对建模数据的一组原子操作的分层查询语言。 建模的数据可以显示在显示单元上。

公开(公告)号：US08352470B2

公开(公告)日：2013-01-08

申请号：US12126831

申请日：2008-05-23

Applicant: Ying Chen ,  Bin He

Inventor： Ying Chen ,  Bin He

IPC: G06F7/00

CPC classification number: G06F17/30412

Abstract: A method for use with an aggregation operation (e.g., on a relational database table) includes a sorting pass and a merging pass. The sorting pass includes: (a) reading blocks of the table from a storage medium into a memory using an aggregation method until the memory is substantially full or until all the data have been read into the memory; (b) determining a number k of blocks to write back to the storage medium from the memory; (c) selecting k blocks from memory, sorting the k blocks, and then writing the k blocks back to the storage medium as a new sublist; and (d) repeating steps (a), (b), and (c) for any unprocessed tuples in the database table. The merging pass includes: merging all the sublists to form an aggregation result using a merge-sort algorithm.

Abstract translation: 用于聚合操作（例如，在关系数据库表中）的方法包括排序通行证和合并通行证。 排序通行证包括：（a）使用聚合方法从存储介质将存储介质的块读取到存储器中，直到存储器基本为满或直到所有数据已被读入存储器为止; （b）从所述存储器确定要从所述存储介质写回的块数k; （c）从存储器中选择k个块，对k个块进行排序，然后将k个块作为新的子列表写入存储介质; 和（d）对数据库表中的任何未处理的元组重复步骤（a），（b）和（c）。 合并通过包括：使用合并排序算法合并所有子列表以形成聚合结果。

公开(公告)号：US08244724B2

公开(公告)日：2012-08-14

申请号：US12776779

申请日：2010-05-10

Applicant: Ying Chen ,  Bin He ,  William Scott Spangler

Inventor： Ying Chen ,  Bin He ,  William Scott Spangler

IPC: G06F17/30

CPC classification number: G06F17/30707

Abstract: One embodiment is a computer-implemented method for classifying documents in a collection of documents according to their intended readerships. The method comprises using a computer to select a document in the collection of documents; and using a computer to determine a characteristic of the selected document, the characteristic being: misleading when the document includes one or more features that are determined to be for a purpose other than reading the document; commercial when the document includes features that are presented for a commercial purpose; or personal when the document includes features of a personal opinion. The method further includes using a computer to classify the selected document as misleading, commercial, or personal according to its determined characteristic; and using a computer to repeat the steps of select document, determine a characteristic of the selected document, and classify the selected document for additional documents in the collection. At least some documents are classified as misleading, at least some documents are classified as commercial, and at least some documents are classified as personal. Other methods and computer program products are also disclosed according to even more embodiments.

Abstract translation: 一个实施例是一种计算机实现的方法，用于根据其预期的读者对文档集合中的文档进行分类。 该方法包括使用计算机在文档集合中选择文档; 并且使用计算机来确定所选择的文档的特征，其特征是：当所述文档包括被确定为用于除了阅读所述文档之外的目的的一个或多个特征时具有误导性; 当文档包含为商业目的呈现的特征时，商业广告; 或个人，当文件包含个人意见的特征。 该方法还包括使用计算机根据其确定的特征将所选择的文档分类为误导，商业或个人; 并使用计算机重复选择文档的步骤，确定所选文档的特征，并将所选择的文档分类到集合中的附加文档。 至少有些文件被归类为误导性的，至少有些文件被归类为商业，至少有一些文件被归类为个人。 根据甚至更多的实施例还公开了其它方法和计算机程序产品。

公开(公告)号：US20090292704A1

公开(公告)日：2009-11-26

申请号：US12126831

申请日：2008-05-23

Applicant: Ying Chen ,  Bin He

Inventor： Ying Chen ,  Bin He

IPC: G06F7/00 ,  G06F17/30

CPC classification number: G06F17/30412

Abstract: A method for use with an aggregation operation (e.g., on a relational database table) includes a sorting pass and a merging pass. The sorting pass includes: (a) reading blocks of the table from a storage medium into a memory using an aggregation method until the memory is substantially full or until all the data have been read into the memory; (b) determining a number k of blocks to write back to the storage medium from the memory; (c) selecting k blocks from memory, sorting the k blocks, and then writing the k blocks back to the storage medium as a new sublist; and (d) repeating steps (a), (b), and (c) for any unprocessed tuples in the database table. The merging pass includes: merging all the sublists to form an aggregation result using a merge-sort algorithm.

Abstract translation: 用于聚合操作（例如，在关系数据库表中）的方法包括排序通行证和合并通行证。 排序通行证包括：（a）使用聚合方法从存储介质将存储介质的块读取到存储器中，直到存储器基本为满或直到所有数据已被读入存储器为止; （b）从所述存储器确定要从所述存储介质写回的块数k; （c）从存储器中选择k个块，对k个块进行排序，然后将k个块作为新的子列表写入存储介质; 和（d）对数据库表中的任何未处理的元组重复步骤（a），（b）和（c）。 合并通过包括：使用合并排序算法合并所有子列表以形成聚合结果。