公开(公告)号：US20110029569A1

公开(公告)日：2011-02-03

申请号：US12871882

申请日：2010-08-30

申请人： Amit GANESH ,  Vikram KAPOOR ,  Vineet MARWAH ,  Kam SHERGILL ,  Roger MacNICOL ,  Sachin KULKARNI ,  Jesse KAMP

发明人： Amit GANESH ,  Vikram KAPOOR ,  Vineet MARWAH ,  Kam SHERGILL ,  Roger MacNICOL ,  Sachin KULKARNI ,  Jesse KAMP

IPC分类号： G06F17/30

CPC分类号： G06F17/30595

摘要： Techniques for storing and manipulating tabular data are provided. According to one embodiment, a user may control whether tabular data is stored in row-level or column-major format. Furthermore, the user may control the level of data compression to achieve an optimal balance between query performance and compression ratios. Tabular data from within the same table may be stored in both column-major and row-major format and compressed at different levels. In addition, tabular data can migrate between column-major format and row-major format in response to various events. For example, in response to a request to update or lock a row stored in column-major format, the row may be migrated and subsequently stored into row-major format. In one embodiment, table partitions are used to enhance data compression techniques. For example, compression tests are performed on a representative table partition, and a compression map is generated and applied to other table partitions.

摘要翻译： 提供了存储和操作表格数据的技术。 根据一个实施例，用户可以控制表格数据是以行主列还是列主格式存储。 此外，用户可以控制数据压缩的水平，以实现查询性能和压缩比之间的最佳平衡。 来自同一表格的表格数据可以以列主和行主格式存储，并在不同级别进行压缩。 此外，响应于各种事件，表格数据可以在列主格式和行主格式之间迁移。 例如，响应于更新或锁定以列主格式存储的行的请求，该行可以被迁移并且随后被存储为行主格式。 在一个实施例中，表分区用于增强数据压缩技术。 例如，压缩测试在代表性的表格分区上执行，生成压缩映射并将其应用于其他表格分区。

公开(公告)号：US08645337B2

公开(公告)日：2014-02-04

申请号：US12769205

申请日：2010-04-28

申请人： Vikram Kapoor ,  Amit Ganesh ,  Jesse Kamp ,  Sachin Kulkarni ,  Vineet Marwah ,  Kam Shergill ,  Roger Macnicol ,  Manosiz Bhattacharyya

发明人： Vikram Kapoor ,  Amit Ganesh ,  Jesse Kamp ,  Sachin Kulkarni ,  Vineet Marwah ,  Kam Shergill ,  Roger Macnicol ,  Manosiz Bhattacharyya

IPC分类号： G06F7/00 ,  G06F17/00 ,  G06F17/30

CPC分类号： G06F17/30153 ,  G06F17/30501 ,  G06F17/30955

摘要： A database server stores compressed units in data blocks of a database. A table (or data from a plurality of rows thereof) is first compressed into a “compression unit” using any of a wide variety of compression techniques. The compression unit is then stored in one or more data block rows across one or more data blocks. As a result, a single data block row may comprise compressed data for a plurality of table rows, as encoded within the compression unit. Storage of compression units in data blocks maintains compatibility with existing data block-based databases, thus allowing the use of compression units in preexisting databases without modification to the underlying format of the database. The compression units may, for example, co-exist with uncompressed tables. Various techniques allow a database server to optimize access to data in the compression unit, so that the compression is virtually transparent to the user.

摘要翻译： 数据库服务器将压缩单位存储在数据库的数据块中。 使用各种各样的压缩技术中的任一种，首先将表（或其多行的数据）压缩为“压缩单位”。 然后，压缩单元被存储在跨越一个或多个数据块的一个或多个数据块行。 结果，单个数据块行可以包括在压缩单元内编码的多个表行的压缩数据。 数据块中的压缩单元的存储与现有的基于数据块的数据库保持兼容，从而允许在预先存在的数据库中使用压缩单元，而无需修改数据库的底层格式。 压缩单元可以例如与未压缩的表共存。 各种技术允许数据库服务器优化对压缩单元中的数据的访问，使得压缩对于用户实际上是透明的。

公开(公告)号：US20120143833A1

公开(公告)日：2012-06-07

申请号：US13371354

申请日：2012-02-10

申请人： Amit Ganesh ,  Vikram Kapoor ,  Vineet Marwah ,  Kam Shergill ,  Roger MacNicol ,  Sachin Kulkarni ,  Jesse Kamp

发明人： Amit Ganesh ,  Vikram Kapoor ,  Vineet Marwah ,  Kam Shergill ,  Roger MacNicol ,  Sachin Kulkarni ,  Jesse Kamp

IPC分类号： G06F7/00

CPC分类号： G06F16/221

摘要： A highly flexible and extensible structure is provided for physically storing tabular data. The structure, referred to as a compression unit, may be used to store tabular data that logically resides in any type of table-like structure. According to one embodiment, compression units are recursive. Thus, a compression unit may have a “parent” compression unit to which it belongs, and may have one or more “child” compression units that belong to it. In one embodiment, compression units include metadata that indicates how the tabular data is stored within them. The metadata for a compression unit may indicate, for example, whether the data is stored in row-major or column major-format the order of the columns within the compression unit (which may differ from the logical order of the columns dictated by the definition of their logical container), a compression technique for the compression unit, the child compression units (if any), etc.

摘要翻译： 提供了高度灵活和可扩展的结构，用于物理存储表格数据。 称为压缩单元的结构可用于存储逻辑上位于任何类型的类似桌面结构的表格数据。 根据一个实施例，压缩单元是递归的。 因此，压缩单元可以具有它所属的“父”压缩单元，并且可以具有属于它的一个或多个“子”压缩单元。 在一个实施例中，压缩单元包括指示表格数据如何被存储在其中的元数据。 压缩单元的元数据例如可以指示数据是以行主列还是列主格式存储在压缩单元内的列的顺序（其可以不同于由定义指定的列的逻辑顺序 的逻辑容器），压缩单元的压缩技术，子压缩单元（如果有的话）等等

公开(公告)号：US20100281004A1

公开(公告)日：2010-11-04

申请号：US12769205

申请日：2010-04-28

申请人： Vikram Kapoor ,  Amit Ganesh ,  Jesse Kamp ,  Sachin Kulkarni ,  Vineet Marwah ,  Kam Shergill ,  Roger Macnicol ,  Manosiz Bhattacharyya

发明人： Vikram Kapoor ,  Amit Ganesh ,  Jesse Kamp ,  Sachin Kulkarni ,  Vineet Marwah ,  Kam Shergill ,  Roger Macnicol ,  Manosiz Bhattacharyya

IPC分类号： G06F17/30

CPC分类号： G06F17/30153 ,  G06F17/30501 ,  G06F17/30955

摘要： A database server stores compressed units in data blocks of a database. A table (or data from a plurality of rows thereof) is first compressed into a “compression unit” using any of a wide variety of compression techniques. The compression unit is then stored in one or more data block rows across one or more data blocks. As a result, a single data block row may comprise compressed data for a plurality of table rows, as encoded within the compression unit. Storage of compression units in data blocks maintains compatibility with existing data block-based databases, thus allowing the use of compression units in preexisting databases without modification to the underlying format of the database. The compression units may, for example, co-exist with uncompressed tables. Various techniques allow a database server to optimize access to data in the compression unit, so that the compression is virtually transparent to the user.

摘要翻译： 数据库服务器将压缩单位存储在数据库的数据块中。 使用各种各样的压缩技术中的任一种，首先将表（或其多行的数据）压缩为“压缩单元”。 然后，压缩单元被存储在跨越一个或多个数据块的一个或多个数据块行。 结果，单个数据块行可以包括在压缩单元内编码的多个表行的压缩数据。 数据块中的压缩单元的存储与现有的基于数据块的数据库保持兼容，从而允许在预先存在的数据库中使用压缩单元，而无需修改数据库的底层格式。 压缩单元可以例如与未压缩的表共存。 各种技术允许数据库服务器优化对压缩单元中的数据的访问，使得压缩对于用户实际上是透明的。

公开(公告)号：US08583692B2

公开(公告)日：2013-11-12

申请号：US12871882

申请日：2010-08-30

申请人： Amit Ganesh ,  Vikram Kapoor ,  Vineet Marwah ,  Kam Shergill ,  Roger MacNicol ,  Sachin Kulkarni ,  Jesse Kamp

发明人： Amit Ganesh ,  Vikram Kapoor ,  Vineet Marwah ,  Kam Shergill ,  Roger MacNicol ,  Sachin Kulkarni ,  Jesse Kamp

IPC分类号： G06F17/30

CPC分类号： G06F17/30595

摘要： Techniques for storing and manipulating tabular data are provided. According to one embodiment, a user may control whether tabular data is stored in row-level or column-major format. Furthermore, the user may control the level of data compression to achieve an optimal balance between query performance and compression ratios. Tabular data from within the same table may be stored in both column-major and row-major format and compressed at different levels. In addition, tabular data can migrate between column-major format and row-major format in response to various events. For example, in response to a request to update or lock a row stored in column-major format, the row may be migrated and subsequently stored into row-major format. In one embodiment, table partitions are used to enhance data compression techniques. For example, compression tests are performed on a representative table partition, and a compression map is generated and applied to other table partitions.

摘要翻译： 提供了存储和操作表格数据的技术。 根据一个实施例，用户可以控制表格数据是以行主列还是列主格式存储。 此外，用户可以控制数据压缩的水平，以实现查询性能和压缩比之间的最佳平衡。 来自同一表格的表格数据可以以列主和行主格式存储，并在不同级别进行压缩。 此外，响应于各种事件，表格数据可以在列主格式和行主格式之间迁移。 例如，响应于更新或锁定以列主格式存储的行的请求，该行可以被迁移并且随后被存储为行主格式。 在一个实施例中，表分区用于增强数据压缩技术。 例如，压缩测试在代表性的表格分区上执行，生成压缩映射并将其应用于其他表格分区。

公开(公告)号：US08521784B2

公开(公告)日：2013-08-27

申请号：US12871882

申请日：2010-08-30

申请人： Amit Ganesh ,  Vikram Kapoor ,  Vineet Marwah ,  Kam Shergill ,  Roger MacNicol ,  Sachin Kulkarni ,  Jesse Kamp

发明人： Amit Ganesh ,  Vikram Kapoor ,  Vineet Marwah ,  Kam Shergill ,  Roger MacNicol ,  Sachin Kulkarni ,  Jesse Kamp

IPC分类号： G06F17/30

摘要： Techniques for storing and manipulating tabular data are provided. According to one embodiment, a user may control whether tabular data is stored in row-level or column-major format. Furthermore, the user may control the level of data compression to achieve an optimal balance between query performance and compression ratios. Tabular data from within the same table may be stored in both column-major and row-major format and compressed at different levels. In addition, tabular data can migrate between column-major format and row-major format in response to various events. For example, in response to a request to update or lock a row stored in column-major format, the row may be migrated and subsequently stored into row-major format. In one embodiment, table partitions are used to enhance data compression techniques. For example, compression tests are performed on a representative table partition, and a compression map is generated and applied to other table partitions.

公开(公告)号：US20120054225A1

公开(公告)日：2012-03-01

申请号：US12871824

申请日：2010-08-30

申请人： Vineet Marwah ,  Vikram Kapoor ,  Jesse Kamp ,  Kam Shergill ,  Roger Macnicol ,  Sachin Kulkami ,  Amit Ganesh

发明人： Vineet Marwah ,  Vikram Kapoor ,  Jesse Kamp ,  Kam Shergill ,  Roger Macnicol ,  Sachin Kulkami ,  Amit Ganesh

IPC分类号： G06F17/30

CPC分类号： G06F17/30442 ,  G06F17/30477 ,  G06F17/30522

摘要： A method and apparatus is provided for optimizing queries received by a database system that relies on an intelligent data storage server to manage storage for the database system. Storing compression units in hybrid columnar format, the storage manager evaluates simple predicates and only returns data blocks containing rows that satisfy those predicates. The returned data blocks are not necessarily stored persistently on disk. That is, the storage manager is not limited to returning disc block images. The hybrid columnar format enables optimizations that provide better performance when processing typical database workloads including both fetching rows by identifier and performing table scans.

摘要翻译： 提供了一种方法和装置，用于优化由数据库系统接收的依赖于智能数据存储服务器来管理数据库系统的存储的查询。 以混合列格式存储压缩单元，存储管理器将评估简单谓词，并仅返回包含满足这些谓词的行的数据块。 返回的数据块不一定存储在磁盘上。 也就是说，存储管理器不限于返回盘块图像。 混合柱状格式可以实现在处理典型数据库工作负载时提供更好性能的优化，包括通过标识符提取行和执行表扫描。

公开(公告)号：US08935223B2

公开(公告)日：2015-01-13

申请号：US12617669

申请日：2009-11-12

申请人： Amit Ganesh ,  Vikram Kapoor ,  Vineet Marwah ,  Kam Shergill ,  Roger MacNicol ,  Sachin Kulkarni ,  Jesse Kamp

发明人： Amit Ganesh ,  Vikram Kapoor ,  Vineet Marwah ,  Kam Shergill ,  Roger MacNicol ,  Sachin Kulkarni ,  Jesse Kamp

IPC分类号： G06F17/30 ,  G06F7/00

CPC分类号： G06F17/30315

摘要： A highly flexible and extensible structure is provided for physically storing tabular data. The structure, referred to as a compression unit, may be used to store tabular data that logically resides in any type of table-like structure. According to one embodiment, compression units are recursive. Thus, a compression unit may have a “parent” compression unit to which it belongs, and may have one or more “child” compression units that belong to it. In one embodiment, compression units include metadata that indicates how the tabular data is stored within them. The metadata for a compression unit may indicate, for example, whether the data is stored in row-major or column major-format, the order of the columns within the compression unit (which may differ from the logical order of the columns dictated by the definition of their logical container), a compression technique for the compression unit, the child compression units (if any), etc.

摘要翻译： 提供了高度灵活和可扩展的结构，用于物理存储表格数据。 称为压缩单元的结构可用于存储逻辑上位于任何类型的类似桌面结构的表格数据。 根据一个实施例，压缩单元是递归的。 因此，压缩单元可以具有它所属的“父”压缩单元，并且可以具有属于它的一个或多个“子”压缩单元。 在一个实施例中，压缩单元包括指示表格数据如何被存储在其中的元数据。 压缩单元的元数据例如可以指示数据是以主要还是列主格式存储的，压缩单元内的列的顺序（其可以不同于由 其逻辑容器的定义），压缩单元的压缩技术，子压缩单元（如果有的话）等等

公开(公告)号：US20120117038A1

公开(公告)日：2012-05-10

申请号：US13296435

申请日：2011-11-15

申请人： Amit Ganesh ,  Vikram Kapoor ,  Vineet Marwah ,  Kam Shergill ,  Roger MacNicol ,  Sachin Kulkarni ,  Jesse Kamp

发明人： Amit Ganesh ,  Vikram Kapoor ,  Vineet Marwah ,  Kam Shergill ,  Roger MacNicol ,  Sachin Kulkarni ,  Jesse Kamp

IPC分类号： G06F7/00

CPC分类号： G06F16/221

摘要： A highly flexible and extensible structure is provided for physically storing tabular data. The structure, referred to as a compression unit, may be used to physically store tabular data that logically resides in any type of table-like structure. Techniques are employed to avoid changing tabular data within existing compression units. Deleting tabular data within compression units is avoided by merely tracking deletion requests, without actually deleting the data. Inserting new tabular data into existing compression units is avoided by storing the new data external to the compression units. If the number of deletions exceeds a threshold, and/or the number of new inserts exceeds a threshold, new compression units may be generated. When new compression units are generated, the previously-existing compression units may be discarded to reclaim storage, or retained to allow reconstruction of prior states of the tabular data.

摘要翻译： 提供了高度灵活和可扩展的结构，用于物理存储表格数据。 称为压缩单元的结构可以用于物理地存储逻辑上驻留在任何类型的类似桌面结构中的表格数据。 采用技术来避免在现有压缩单元内改变表格数据。 只有跟踪删除请求才能避免在压缩单元内删除表格数据，而不会实际删除数据。 通过将新的数据存储在压缩单元外部来避免将新的表格数据插入现有的压缩单元。 如果删除的数量超过阈值，和/或新插入的数量超过阈值，则可能产生新的压缩单元。 当生成新的压缩单元时，先前存在的压缩单元可能被丢弃以回收存储，或保留以允许重建表格数据的先前状态。

公开(公告)号：US20100281079A1

公开(公告)日：2010-11-04

申请号：US12769508

申请日：2010-04-28

申请人： Vineet Marwah ,  Vikram Kapoor ,  Amit Ganesh ,  Jesse Kamp ,  Sachin Kulkarni ,  Roger Macnicol ,  Kam Shergill ,  Manosiz Bhattacharyya

发明人： Vineet Marwah ,  Vikram Kapoor ,  Amit Ganesh ,  Jesse Kamp ,  Sachin Kulkarni ,  Roger Macnicol ,  Kam Shergill ,  Manosiz Bhattacharyya

IPC分类号： G06F17/30

CPC分类号： H03M7/30 ,  G06F17/30595

摘要： Techniques are described herein for automatically selecting the compression techniques to be used on tabular data. A compression analyzer gives users high-level control over the selection process without requiring the user to know details about the specific compression techniques that are available to the compression analyzer. Users are able to specify, for a given set of data, a “balance point” along the spectrum between “maximum performance” and “maximum compression”. The point thus selected is used by the compression analyzer in a variety of ways. For example, in one embodiment, the compression analyzer uses the user-specified balance point to determine which of the available compression techniques qualify as “candidate techniques” for the given set of data. The compression analyzer selects the compression technique to use on a set of data by actually testing the candidate compression techniques against samples from the set of data. After testing the candidate compression techniques against the samples, the resulting compression ratios are compared. The compression technique to use on the set of data is then selected based, in part, on the compression ratios achieved during the compression tests performed on the sample data.

摘要翻译： 这里描述了用于自动选择要在表格数据上使用的压缩技术的技术。 压缩分析仪为用户提供了对选择过程的高级控制，而不需要用户了解有关压缩分析器可用的特定压缩技术的细节。 用户可以为给定的数据集指定沿“最大性能”和“最大压缩”之间的“平衡点”。 所选择的点由压缩分析器以各种方式使用。 例如，在一个实施例中，压缩分析器使用用户指定的平衡点来确定哪个可用的压缩技术符合给定的数据集合的“候选技术”。 压缩分析仪通过对来自该组数据的样本实际测试候选压缩技术来选择对一组数据使用的压缩技术。 在针对样品测试候选压缩技术之后，比较所得到的压缩比。 然后，部分地基于在对样本数据执行的压缩测试期间实现的压缩比来选择在该组数据上使用的压缩技术。