-
公开(公告)号:US20100011031A1
公开(公告)日:2010-01-14
申请号:US12554541
申请日:2009-09-04
申请人: Wei Huang , Yizheng Zhou , Bin Yu , Wenting Tang , Christian F. Beedgen
发明人: Wei Huang , Yizheng Zhou , Bin Yu , Wenting Tang , Christian F. Beedgen
CPC分类号: H04L63/1425 , G06F11/3476 , G06F17/30595 , G06F21/552 , G06F2201/86 , G06F2221/034 , H04L63/1408
摘要: A logging system includes an event receiver and a storage manager. The receiver receives log data, processes it, and outputs a column-based data “chunk.” The manager receives and stores chunks. The receiver includes buffers that store events and a metadata structure that stores metadata about the contents of the buffers. Each buffer is associated with a particular event field and includes values from that field from one or more events. The metadata includes, for each “field of interest,” a minimum value and a maximum value that reflect the range of values of that field over all of the events in the buffers. A chunk is generated for each buffer and includes the metadata structure and a compressed version of the buffer contents. The metadata structure acts as a search index when querying event data. The logging system can be used in conjunction with a security information/event management (SIEM) system.
摘要翻译: 记录系统包括事件接收器和存储管理器。 接收器接收日志数据,处理它,并输出基于列的数据“块”。 经理收到并存储块。 接收器包括存储事件的缓冲器和存储关于缓冲器的内容的元数据的元数据结构。 每个缓冲区与一个特定事件字段相关联,并包含一个或多个事件的该字段的值。 对于每个“感兴趣的领域”,元数据包括反映缓冲器中的所有事件的该字段的值的范围的最小值和最大值。 为每个缓冲区生成一个块,并包括元数据结构和缓冲区内容的压缩版本。 元数据结构在查询事件数据时用作搜索索引。 记录系统可以与安全信息/事件管理(SIEM)系统结合使用。
-
公开(公告)号:US20140195502A1
公开(公告)日:2014-07-10
申请号:US14237280
申请日:2012-08-24
申请人: Wei Huang , Yizheng Zhou , Bin Yu
发明人: Wei Huang , Yizheng Zhou , Bin Yu
IPC分类号: G06F17/30
CPC分类号: G06F16/211 , G06F16/1744 , G06F16/221 , G06F16/245 , H04L63/1425
摘要: A data storage system includes a storage engine to partition data across multiple dimensions. The storage engine determines chunks according to the partitioning, and performs column-based storage of the chunks.
摘要翻译: 数据存储系统包括用于跨多个维度分割数据的存储引擎。 存储引擎根据分区确定块,并执行块的基于列的存储。
-
公开(公告)号:US09009139B2
公开(公告)日:2015-04-14
申请号:US13699953
申请日:2011-06-10
申请人: Wei Huang , Yizheng Zhou , Bin Yu
发明人: Wei Huang , Yizheng Zhou , Bin Yu
IPC分类号: G06F17/30
CPC分类号: G06F17/30563 , G06F17/30442
摘要: A query pipeline is created (514) from a query request. The query pipeline includes multiple query operations including multiple query operators. A first query operator and a second query operator perform first and second query operations on a database (526) and on data outside the database (534). A result from the first query operation in the query pipeline is fed to the second query operation in the query pipeline.
摘要翻译: 从查询请求创建查询流水线(514)。 查询流水线包括多个查询操作,包括多个查询运算符。 第一查询运算符和第二查询运算符对数据库(526)和数据库外的数据(534)执行第一和第二查询操作。 查询流水线中的第一个查询操作的结果被馈送到查询流水线中的第二个查询操作。
-
公开(公告)号:US09166989B2
公开(公告)日:2015-10-20
申请号:US12554541
申请日:2009-09-04
申请人: Wei Huang , Yizheng Zhou , Bin Yu , Wenting Tang , Christian F. Beedgen
发明人: Wei Huang , Yizheng Zhou , Bin Yu , Wenting Tang , Christian F. Beedgen
CPC分类号: H04L63/1425 , G06F11/3476 , G06F17/30595 , G06F21/552 , G06F2201/86 , G06F2221/034 , H04L63/1408
摘要: A logging system includes an event receiver and a storage manager. The receiver receives log data, processes it, and outputs a column-based data “chunk.” The manager receives and stores chunks. The receiver includes buffers that store events and a metadata structure that stores metadata about the contents of the buffers. Each buffer is associated with a particular event field and includes values from that field from one or more events. The metadata includes, for each “field of interest,” a minimum value and a maximum value that reflect the range of values of that field over all of the events in the buffers. A chunk is generated for each buffer and includes the metadata structure and a compressed version of the buffer contents. The metadata structure acts as a search index when querying event data. The logging system can be used in conjunction with a security information/event management (SIEM) system.
摘要翻译: 记录系统包括事件接收器和存储管理器。 接收器接收日志数据,处理它,并输出基于列的数据“块”。管理器接收并存储块。 接收器包括存储事件的缓冲器和存储关于缓冲器的内容的元数据的元数据结构。 每个缓冲区与一个特定事件字段相关联,并包含一个或多个事件的该字段的值。 对于每个“感兴趣的领域”,元数据包括反映缓冲器中所有事件的该字段的值的范围的最小值和最大值。 为每个缓冲区生成一个块,并包括元数据结构和缓冲区内容的压缩版本。 元数据结构在查询事件数据时用作搜索索引。 记录系统可以与安全信息/事件管理(SIEM)系统结合使用。
-
公开(公告)号:US20130073573A1
公开(公告)日:2013-03-21
申请号:US13699953
申请日:2011-06-10
申请人: Wei Huang , Yizheng Zhou , Bin Yu
发明人: Wei Huang , Yizheng Zhou , Bin Yu
IPC分类号: G06F17/30
CPC分类号: G06F17/30563 , G06F17/30442
摘要: A query pipeline is created (514) from a query request. The query pipeline includes multiple query operations including multiple query operators. A first query operator and a second query operator perform first and second query operations on a database (526) and on data outside the database (534). A result from the first query operation in the query pipeline is fed to the second query operation in the query pipeline.
摘要翻译: 从查询请求创建查询流水线(514)。 查询流水线包括多个查询操作,包括多个查询运算符。 第一查询运算符和第二查询运算符对数据库(526)和数据库外的数据(534)执行第一和第二查询操作。 查询流水线中的第一个查询操作的结果被馈送到查询流水线中的第二个查询操作。
-
公开(公告)号:US20140244650A1
公开(公告)日:2014-08-28
申请号:US14352121
申请日:2011-12-20
IPC分类号: G06F17/30
CPC分类号: G06F16/285 , G06F16/2272 , H04L43/04 , H04L43/12
摘要: A distributed event processing method includes providing a plurality of connectors. Each provided connector is configured to acquire event data from an assigned data source, partition acquired event data into clusters, and divide each cluster into chunks. The method also includes collecting the chunks from the plurality of connectors and storing the chunks to a data file that can be queried.
摘要翻译: 分布式事件处理方法包括提供多个连接器。 每个提供的连接器被配置为从分配的数据源获取事件数据,将获取的事件数据分成簇,并将每个簇划分成块。 该方法还包括从多个连接器收集块,并将块存储到可被查询的数据文件中。
-
公开(公告)号:US20140280075A1
公开(公告)日:2014-09-18
申请号:US14237192
申请日:2012-08-24
申请人: Wei Huang , Yizheng Zhou
发明人: Wei Huang , Yizheng Zhou
IPC分类号: G06F17/30
CPC分类号: G06F16/23 , G06F16/2455 , G06F16/278 , G06F16/283 , H04L41/069 , H04L63/1425
摘要: A data storage system includes a partitioning module to partition data across multiple dimensions simultaneously. The partitioning may be based on a sizing parameter for each dimension. The partitioning module stores a cluster including the partitioned event data and metadata including attributes identifying the cluster.
摘要翻译: 数据存储系统包括分区模块,用于同时跨多个维度划分数据。 分区可以基于每个维度的大小参数。 分区模块存储包括分区事件数据和元数据的集群,其中包括标识集群的属性。
-
-
-
-
-
-
-