Abstract:
A low-overhead merge technique enables restart of a merge operation with minimal logging of state information relating to progress of the merge operation by a volume layer of a storage input/output (I/O) stack executing on one or more nodes of a cluster. The technique enables restart of the merge operation by ensuring that metadata, i.e., metadata pages, generated during the merge operation is not subject to de-duplication by providing a unique value in each metadata page that distinguishes the page, i.e., renders the page distinct or “unique”, from other metadata pages in an extent store. In addition, the technique ensures that a reference count on each metadata page is a value denoting a lack of de-duplication. To that end, the extent store layer is configured to not increment the reference count for a metadata page if, during the merge operation, the page is identical (and thus subject to deduplication) to an existing metadata page in the extent store.
Abstract:
A method performed in a system that has a plurality of volumes stored to storage hardware, the method including generating, for each of the volumes, a respective space saving potential iteratively over time and scheduling space saving operations among the plurality of volumes by analyzing each of the volumes for space saving potential and assigning priority of resources based at least in part on space saving potential.
Abstract:
In one embodiment, a node coupled to one or more storage devices executes a storage input/output (I/O) stack having a volume layer. The volume layer manages volume metadata embodied as mappings from offsets of a logical unit (LUN) to extent keys associated with storage locations for extents on the one or more storage devices. Volume metadata is maintained as a dense tree metadata structure representing successive points in time. The dense tree metadata structure has multiple levels, wherein a top level of the dense tree metadata structure represents newer volume metadata changes and descending levels of the dense tree metadata structure represent older volume metadata changes. The node accesses a latest version of changes to the volume metadata by searching from the top level to the descending levels in the dense tree metadata structure.
Abstract:
A method for sharing data blocks in a hierarchical file system in a storage server includes allocating a plurality of data blocks in the file system, and sharing data blocks in the file system, without using a persistent point-in-time image, to avoid duplication of data blocks. A method for identifying data blocks that can be shared includes computing a fingerprint for each of multiple data blocks to be written to a storage facility and storing the fingerprint with information identifying the data block in an entry in a set of metadata. The set of metadata is used to identify data blocks which are duplicates.
Abstract:
A system can maintain multiple queues for deduplication requests of different priorities. The system can also designate priority of storage units. The scheduling priority of a deduplication request is based on the priority of the storage unit indicated in the deduplication request and a trigger for the deduplication request.
Abstract:
In one embodiment, a node of a cluster executing a storage input/output (I/O) stack having a volume layer, stores a multi-level dense tree metadata structure. Each level of the dense tree metadata structure includes volume metadata entries for storing volume metadata. One or more non-volatile logs (NVLogs) are updated. The one or more NVLogs including a volume layer log configured to record changes to the volume metadata, wherein volume metadata entries inserted into a top-level of the dense tree metadata structure are recorded in the volume layer log. The node writes volume metadata entries from the volume layer log to one or more storage devices to be stored as extents.
Abstract:
A storage appliance arranges snapshot data and snapshot metadata into different structures, and arranges the snapshot metadata to facilitate efficient snapshot manipulation, which may be for snapshot management or snapshot restore. The storage appliance receives snapshots according to a forever incremental configuration and arranges snapshot metadata into different types of records. The storage appliance stores these records in key-value stores maintained for each defined data collection (e.g., volume). The storage appliance arranges the snapshot metadata into records for inode information, records for directory information, and records that map source descriptors of data blocks to snapshot file descriptors. The storage appliance uses a locally generated snapshot identifier as a key prefix for the records to conform to a sort constrain of the key-value store, which allows the efficiency of the key-value store to be leveraged. The snapshot metadata arrangement facilitates efficient snapshot restore, file restore, and snapshot reclamation.
Abstract:
Systems and methods for creation and retention of immutable snapshots to facilitate ransomware protection are provided. According to one embodiment, multiple use cases for retention of snapshots are supported, including (i) maintaining a locked snapshot on a source volume of a first storage system on which it was originally created for at least an associated immutable retention time; (ii) replicating the locked snapshot to a destination volume of a second storage system and also maintaining the replica of the locked snapshot on the destination volume for at least the associated immutable retention time; and (iii) maintaining an unlocked snapshot on the source volume, replicating the unlocked snapshot to the destination volume, locking the replicated snapshot on the destination volume when it has an associated non-zero immutable retention time, and thereafter maintaining the replica on the destination volume in accordance with the immutable retention time.
Abstract:
With a forever incremental snapshot configuration and a typical caching policy (e.g., least recently used), a storage appliance may evict stable data blocks of an older snapshot, perhaps unchanged data blocks of the snapshot baseline. If stable data blocks have been evicted, restore of a recent snapshot will suffer the time penalty of downloading the stable blocks for restoring the recent snapshot. Creating synthetic baseline snapshots and refreshing eviction data of stable data blocks can avoid eviction of stable data blocks and reduce the risk of violating a recovery time objective.
Abstract:
Techniques are provided for incremental backup to an object store. A request may be received from an application to perform a backup from a volume hosted by a node to a backup target within the object store. A set of changed files within the volume since a prior backup of the volume was performed to the backup target is identified, along with metadata associated with the set of changed files. The metadata is utilized to identify changed data blocks comprising data of the set of changed files that was modified since the prior backup. The changed data blocks are backed up to the object store.