MEMORY-FRUGAL INDEX DESIGN IN STORAGE ENGINE

    公开(公告)号:US20240061823A1

    公开(公告)日:2024-02-22

    申请号:US18270830

    申请日:2021-01-04

    IPC分类号: G06F16/22 G06F16/23

    摘要: A storage engine may be configured to employ different formats for index fragments and index entries of respective records in the index fragments based at least in part on record properties of the respective records in the index fragments, to reduce an amount of memory space that is consumed or used for storing the index fragments in a memory associated with the storage engine, without compromising the efficiency of searching the records stored in a storage device. Using different formats for index fragments covering records of different record properties, the storage engine may further be configured to create, maintain, and update index mappings for records stored or included in the storage device, to provide functionalities of point-lookup, range query, deletion, and additions of the records in the storage device.

    END TO END CHECK CODE PROTECTION IN STORAGE ENGINE

    公开(公告)号:US20240126641A1

    公开(公告)日:2024-04-18

    申请号:US18270184

    申请日:2021-01-22

    IPC分类号: G06F11/10

    CPC分类号: G06F11/1004

    摘要: A client device may transmit a data stream including an object (such as a data file or record) to a storage system for storing the object in a storage device. In response to receiving the data stream, the storage system may store or write data of the object into a plurality of logical blocks of the storage device with an end-to-end data protection based at least in part on a comparison of a combination of check codes of a header of the object, the object and padding data with a combination of a plurality of check codes that are generated separately for metadata associated with the object and corresponding parts of the object stored in the plurality of logical blocks. The storage system may further provide an end-to-end data protection for reading data of an object stored in multiple logical blocks of a storage device.

    METHOD AND SYSTEM FOR IN-MEMORY METADATA REDUCTION IN CLOUD STORAGE SYSTEM

    公开(公告)号:US20230342293A1

    公开(公告)日:2023-10-26

    申请号:US18214924

    申请日:2023-06-27

    摘要: Methods and systems for in-memory metadata reduction in cloud storage system are provided. According to an aspect, a method comprises receiving a first command to write a data stream to a storage device; writing the data stream into a plurality of fragments having logical addresses corresponding to physical addresses on the storage device; and generating an index for individual fragment of the plurality of fragments, the index indicating information to locate the physical addresses of the individual fragment. Individual records in the individual fragment have a same pre-set logical size and all individual records in the individual fragment are continuous, and the index indicates the information including at least: an offset value of the individual record in the individual fragment; the pre-set logical size of the individual record; and a pre-set physical size of the individual record.

    Rapid side-channel access to storage devices

    公开(公告)号:US10552332B2

    公开(公告)日:2020-02-04

    申请号:US15976034

    申请日:2018-05-10

    发明人: Yu Du Ping Zhou Shu Li

    摘要: Disclosed are systems, methods, and apparatuses for providing a high-speed data path to storage devices. In one embodiment, a method is disclosed comprising receiving, by the processor, a data access command, the data access command specifying a location in memory to access data; issuing, by the processor, the data access command to the storage device via a first datapath, the first datapath comprising a non-block datapath; and accessing, by the processor, the non-volatile storage component through the first datapath and the memory, wherein the non-volatile storage component of the storage device is mapped to memory accessible by the processor.

    STORAGE RECORD ENGINE IMPLEMENTING EFFICIENT TRANSACTION REPLAY

    公开(公告)号:US20230342367A1

    公开(公告)日:2023-10-26

    申请号:US18215057

    申请日:2023-06-27

    摘要: A storage record engine implemented on a storage system is provided. The storage record engine further organizes hosted storage of the storage system into superblocks and chunks organized by respective metadata, the chunks being further organized into chunk segments amongst superblocks. Persistent storage operations may cause modifications to the metadata, which may be recorded in a transaction log, records of which may be replayed to commit the modifications to hosted storage. The replay functionality may establish recovery of data following a system failure, wherein replay of records of transaction logs in a fashion interleaved with checkpoint metadata avoids preemption of normal storage device activity during a recovery process, and improves responsiveness of the storage system from the perspective of end devices.

    DATA LAYOUT OPTIMIZATION FOR OBJECT-ORIENTED STORAGE ENGINE

    公开(公告)号:US20240086095A1

    公开(公告)日:2024-03-14

    申请号:US18274295

    申请日:2021-02-07

    IPC分类号: G06F3/06

    摘要: A storage engine may be configured to receive data including a plurality of records from a client device, and generate a plurality of record headers for the plurality of records. The storage engine may then transfer a group of record headers of the plurality of record headers to a storage device to cause the storage device to store the group of record headers consecutively in a data sector of the storage device, and further transfer a subset of payload data of one or more records associated with the group of record headers to the storage device to cause the storage device to store the one or more records after the group of record headers in the data sector of the storage device.

    USING OBJECT PROPERTIES FOR FACILITATING OBJECT STORAGE

    公开(公告)号:US20240086091A1

    公开(公告)日:2024-03-14

    申请号:US18274313

    申请日:2021-02-10

    IPC分类号: G06F3/06

    CPC分类号: G06F3/0631 G06F3/0668

    摘要: A storage engine may obtain one or more object access properties of an object to be received, and determine a type of storage device that is suitable or desirable for storing the object from among different types of storage devices based at least in part on the one or more object access properties of the object to be received. In response to determining the type of storage device, the storage engine may allocate a storage device of such type for the object. The storage engine may then receive the object, and store the object into the allocated storage device.

    RECYCLE OPTIMIZATION IN STORAGE ENGINE
    8.
    发明公开

    公开(公告)号:US20240061577A1

    公开(公告)日:2024-02-22

    申请号:US18269918

    申请日:2020-12-28

    IPC分类号: G06F3/06

    摘要: When performing a recycling operation on a storage device, a storage system may use or create a data buffer in the storage device, and designate the data buffer to temporarily store data of data blocks to be recycled in the storage device using direct memory access (DMA) operations that are performed internally in the storage device, without the need of reading the data of the data blocks from the storage device and writing the data into a host memory of the storage system, thereby saving or reducing the consumptions of the communication bandwidth of a communication channel between the storage system and the storage device, and the memory bandwidth of the host memory.

    System and method for managing resources of a storage device and quantifying the cost of I/O requests

    公开(公告)号:US11816043B2

    公开(公告)日:2023-11-14

    申请号:US16964134

    申请日:2018-06-25

    IPC分类号: G06F13/16

    CPC分类号: G06F13/161

    摘要: One embodiment facilitates measurement of a performance of a storage device. During operation, the system determines a normalized cost for an I/O request, wherein the normalized cost is independent of an access pattern and a type of the I/O request, wherein the normalized cost is indicated by a first number of virtual I/O operations consumed by the I/O request, and wherein a virtual I/O operation is used as a logical unit of cost associated with physical I/O operations. The system identifies a performance metric for the storage device by calculating a second number of virtual I/O operations per second which can be executed by the storage device. The system allocates incoming I/O requests to the storage device based on the performance metric, e.g., to satisfy a Quality of Service requirement, thereby causing an enhanced measurement of the performance of the storage device.

    SYSTEM AND METHOD FOR MANAGING RESOURCES OF A STORAGE DEVICE AND QUANTIFYING THE COST OF I/O REQUESTS

    公开(公告)号:US20210365390A1

    公开(公告)日:2021-11-25

    申请号:US16964134

    申请日:2018-06-25

    IPC分类号: G06F13/16

    摘要: One embodiment facilitates measurement of a performance of a storage device. During operation, the system determines a normalized cost for an I/O request, wherein the normalized cost is independent of an access pattern and a type of the I/O request, wherein the normalized cost is indicated by a first number of virtual I/O operations consumed by the I/O request, and wherein a virtual I/O operation is used as a logical unit of cost associated with physical I/O operations. The system identifies a performance metric for the storage device by calculating a second number of virtual I/O operations per second which can be executed by the storage device. The system allocates incoming I/O requests to the storage device based on the performance metric, e.g., to satisfy a Quality of Service requirement, thereby causing an enhanced measurement of the performance of the storage device.