Redo log for append only storage scheme

    公开(公告)号:US11023328B2

    公开(公告)日:2021-06-01

    申请号:US16049279

    申请日:2018-07-30

    摘要: A new snapshot of a storage volume is created by suppressing write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot. Data is written in segments and Metadata is stored in the segments indicating LBAs of data stored therein and offsets within the segments at which data for LBAs are stored. For write requests, index entries are stored in an index buffer for a segment and written to the segment when the buffer is full. A redo entry is created in a redo buffer for multiple storage volumes and slices for each write request. Write requests are acknowledged when the redo buffer is written to redo segments on a storage device. On restart, index buffers are reconstructed from the redo segments if needed.

    Block map cache
    2.
    发明授权

    公开(公告)号:US10976938B2

    公开(公告)日:2021-04-13

    申请号:US16049073

    申请日:2018-07-30

    摘要: A new snapshot of a storage volume is created by suppressing write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. A block map records segments where current data for an LBA of a slice of a storage volume is stored. Block maps may be written to a storage device in order to free memory. Block maps may be read back into memory when needed. Writing and reading of block maps may be performed upon fragments of block maps. On restarting of the storage node, block maps may be restored from block maps stored in the storage device.

    DATA DE-DUPLICATION
    3.
    发明申请
    DATA DE-DUPLICATION 审中-公开

    公开(公告)号:US20200333970A1

    公开(公告)日:2020-10-22

    申请号:US16387400

    申请日:2019-04-17

    IPC分类号: G06F3/06 G06F12/02

    摘要: A new snapshot of a storage volume is created by suppressing write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. An orchestration layer implements a bundled application that is provisioned with storage volumes and containers. A snapshot of the application may be created and used to rollback or clone the application. De-duplication may be performed by creating a signature map and identifying duplicated blocks. Blocks of segments containing duplicated blocks are copied to pool segments and metadata of those segments of the same logical storage unit may be consolidated to pool metadata segments. The identification of duplicate blocks may be performed in a cloud computing platform.

    Storage scheme for a distributed storage system

    公开(公告)号:US10579276B2

    公开(公告)日:2020-03-03

    申请号:US15703803

    申请日:2017-09-13

    IPC分类号: G06F12/00 G06F3/06

    摘要: A storage scheme allocates portions of a logical volume to storage nodes in excess of the capacity of the storage nodes. Slices of the storage nodes and segments of slices are allocated in response to write requests such that actual allocation on the storage nodes is only in response to usage. Segments are identified with virtual segment identifiers that are retained when segments are moved to a different storage node. Logical volumes may therefore be moved seamlessly to different storage nodes to ensure sufficient storage capacity. Data is written to new locations in segments having space and a block map tracks the last segment to which data for a given address is written. Garbage collection is performed to free segments that contain invalid data, i.e. data for addresses that have been subsequently written to.

    Implementing Affinity And Anti-Affinity Constraints In A Bundled Application

    公开(公告)号:US20200042392A1

    公开(公告)日:2020-02-06

    申请号:US16050655

    申请日:2018-07-31

    摘要: A new snapshot of a storage volume is created by suppressing write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. An orchestration layer implements a bundled application that is provisioned with storage volumes and containers. A snapshot of the application may be created and used to rollback or clone the application. The amount of processing cores, memory, and containers may be increased or decreased based on usage. Components of the bundled application may be assigned to nodes to satisfy affinity and anti-affinity rules. Device sets may be generated according to these rules and used to plan for the assignment of storage volumes and containers of the bundled application.

    Creating snapshots of a storage volume in a distributed storage system

    公开(公告)号:US11249851B2

    公开(公告)日:2022-02-15

    申请号:US16562062

    申请日:2019-09-05

    IPC分类号: G06F11/14 G06F3/06

    摘要: A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. A segment maps segments to a particular snapshot and metadata stored in the segment indicates storage volume addresses of data written to the segment. The snapshots may be represented by a storage manager in a hierarchy that identifies an ordering of snapshots and branches to clone snapshots. A non-snapshot volume may be converted to a snapshot volume at any point after creation.

    Creating Snapshots of a Storage Volume in a Distributed Storage System

    公开(公告)号:US20210073079A1

    公开(公告)日:2021-03-11

    申请号:US16562062

    申请日:2019-09-05

    IPC分类号: G06F11/14 G06F3/06

    摘要: A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. A segment maps segments to a particular snapshot and metadata stored in the segment indicates storage volume addresses of data written to the segment. The snapshots may be represented by a storage manager in a hierarchy that identifies an ordering of snapshots and branches to clone snapshots. A non-snapshot volume may be converted to a snapshot volume at any point after creation.

    Implementing storage volumes over multiple tiers

    公开(公告)号:US10599622B2

    公开(公告)日:2020-03-24

    申请号:US16050801

    申请日:2018-07-31

    摘要: A distributed storage system stores a storage volume as segments that are allocated as needed and assigned VSIDs according to a monotonically increasing counter. The storage volume may be provisioned by an orchestration layer that manages the storage volumes as well as containers executing executable components of the storage volume. A storage volume may be stored across multiple tiers with a storage node managing on which tier a segment is stored based on a volume tier policy and usage of the segments. The tiering policy may be used to relocate the entire storage volume to a tier, such as a cloud computing platform, after which an application may be restored on the tier. Segments on a tier with built in redundancy may be used as both primary and backup copies.

    Block Map Cache
    9.
    发明申请
    Block Map Cache 审中-公开

    公开(公告)号:US20200034052A1

    公开(公告)日:2020-01-30

    申请号:US16049073

    申请日:2018-07-30

    摘要: A new snapshot of a storage volume is created by suppressing write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. A block map records segments where current data for an LBA of a slice of a storage volume is stored. Block maps may be written to a storage device in order to free memory. Block maps may be read back into memory when needed. Writing and reading of block maps may be performed upon fragments of block maps. On restarting of the storage node, block maps may be restored from block maps stored in the storage device.

    Storage scheme for a distributed storage system

    公开(公告)号:US10430105B2

    公开(公告)日:2019-10-01

    申请号:US15703719

    申请日:2017-09-13

    IPC分类号: G06F12/02 G06F3/06

    摘要: A storage scheme allocates portions of a logical volume to storage nodes in excess of the capacity of the storage nodes. Slices of the storage nodes and segments of slices are allocated in response to write requests such that actual allocation on the storage nodes is only in response to usage. Segments are identified with virtual segment identifiers that are retained when segments are moved to a different storage node. Logical volumes may therefore be moved seamlessly to different storage nodes to ensure sufficient storage capacity. Data is written to new locations in segments having space and a block map tracks the last segment to which data for a given address is written. Garbage collection is performed to free segments that contain invalid data, i.e. data for addresses that have been subsequently written to.