Hybrid data storage and load system with ROWID lookup

    公开(公告)号:US11514028B2

    公开(公告)日:2022-11-29

    申请号:US16863834

    申请日:2020-04-30

    Applicant: SAP SE

    Abstract: System, method, and various embodiments for providing a hybrid data storage and load system are described herein. An embodiment operates by receiving a lookup request comprising a row identifier (ROWID) corresponding to a row of a table of a database. Which of a first set of pages corresponds to a sorted position of the ROWID is identified. The identified page of the first set of pages is loaded from the disk storage to the memory storage. A sorted row position of the ROWID is identified from a memory block of the loaded page from the first set of pages. One of a second set of pages that includes the sorted row position is identified. The identified one of the second set of pages is loaded from the disk storage into memory. The loaded identified one of the second set of pages is searched for the unsorted position of the ROWID.

    Adaptive caching for hybrid columnar databases with heterogeneous page sizes

    公开(公告)号:US11481134B1

    公开(公告)日:2022-10-25

    申请号:US17328656

    申请日:2021-05-24

    Applicant: SAP SE

    Abstract: Disclosed herein are system, method, and computer program product embodiments for adaptive caching for hybrid columnar databases with heterogeneous page sizes. An embodiment operates by receiving a request to load a new page of memory from a disk in a buffer cache. The embodiment scans one or more pools comprising one or more pages of the same size in a buffer cache. The embodiment determines an increment of a reuse rate for the pools in the buffer cache within a time interval. The embodiment determines a cumulative reuse rate that is the sum of the increments of the reuse rate over several time intervals. The embodiment determines a gliding average reuse rate of the cumulative reuse rate over several time intervals. The embodiment compares the average reuse rates of the plurality of the pools to a threshold to dynamically determine whether a pool should reuse memory from the existing pages of the same pool or rebalance memory from one or more victim pools.

    Space efficient vector for columnar data storage

    公开(公告)号:US11386082B2

    公开(公告)日:2022-07-12

    申请号:US16893697

    申请日:2020-06-05

    Applicant: SAP SE

    Abstract: Disclosed herein are system, method, and computer program product embodiments for providing paged and compressed storage for column data, while maintaining existing access mechanisms for the data. In order to reduce an in-memory footprint for column data, columns may be stored in pageable format using page chains, and only those pages of the column data needed to resolve a query will be placed in memory, and evicted from memory when no longer needed. In order to further reduce the footprint for these columns, compression can be applied, and the compressed column data stored in the same pageable format using page chains. The compressed data includes a plurality of vectors, each of which is converted into pages and stored on the page chain with the others so that they can be efficiently retrieved during database retrieval operations.

    Out of memory error handling for data definition language operations

    公开(公告)号:US11086832B2

    公开(公告)日:2021-08-10

    申请号:US14885752

    申请日:2015-10-16

    Applicant: SAP SE

    Abstract: The subject matter disclosed herein relates to out of memory error handling in a database system. A database operation can be received by an in-memory database. The database operation can be for a database object stored in a database table that can be represented as a plurality of pages that can be persisted in a page chain. The in-memory database can reserve out of memory space sufficient to load a predetermined number of pages in the page chain into memory. The in-memory database can iteratively process each page in the page chain until completion of the database operation. The iterative process can include loading the page from persistence into memory of the in-memory database, performing at least a portion of the database operation using the loaded page, and unloading the page from the memory of the in-memory database. Related apparatus, systems, techniques, and articles are also described.

    Paged column dictionary
    29.
    发明授权

    公开(公告)号:US11537578B2

    公开(公告)日:2022-12-27

    申请号:US15958663

    申请日:2018-04-20

    Applicant: SAP SE

    Abstract: A request for an entry of a dictionary having multiple dictionary blocks may be received, where the dictionary stores string values associated with corresponding value identifiers (IDs) for dictionary compression. One of the dictionary blocks may be selectively loaded into memory, and the dictionary block that has been loaded into memory may be searched. A value ID directory may be constructed in memory, where the value ID directory includes last value IDs for the dictionary blocks, and each of the last value IDs is mapped to an index of one of the dictionary blocks that includes a string value for that last value ID. A separator directory may also be constructed in memory, where the separator directory includes separators for the dictionary blocks, and each of the separators is mapped to an index of one of the dictionary blocks that includes a string value for that separator.

    In-place load unit conversion
    30.
    发明授权

    公开(公告)号:US11372845B2

    公开(公告)日:2022-06-28

    申请号:US16893703

    申请日:2020-06-05

    Applicant: SAP SE

    Abstract: Disclosed herein are system, method, and computer program product embodiments for converting column loadable data in a database into a page loadable format. When a column oriented in-memory database needs to perform an operation on a table, the affected columns must be loaded fully into memory. In some cases, a database may partition tables, allowing for column fragments corresponding to those partitions to be loaded into memory. These column fragments may exceed available memory, such that it is beneficial to convert the column fragment into a page loadable format. Approaches are discussed herein for performing the conversion without the need to update global metadata during the process, allowing for multiple parallel column fragments to be updated, allowing data manipulation language (DML) operations to continue on the column during the conversion process, and reducing the impact of blocked parallel savepoint operations during the conversion process by chunking the conversion process.

Patent Agency Ranking