Abstract:
Techniques are provided for providing a storage abstraction layer for a composite aggregate architecture. A storage abstraction layer is utilized as an indirection layer between a file system and a storage environment. The storage abstraction layer obtains characteristic of a plurality of storage providers that provide access to heterogeneous types of storage of the storage environment (e.g., solid state storage, high availability storage, object storage, hard disk drive storage, etc.). The storage abstraction layer generates storage bins to manage storage of each storage provider. The storage abstraction layer generates a storage aggregate from the heterogeneous types of storage as a single storage container. The storage aggregate is exposed to the file system as the single storage container that abstracts away from the file system the management and physical storage details of data of the storage aggregate.
Abstract:
Techniques to clone a writeable data object in non-persistent memory are disclosed. The writeable data object is stored in a storage structure in non-persistent memory that corresponds to a portion of a persistent storage. The techniques enable cloning of the writeable data object without having to wait until the writeable data object is saved to the persistent storage and without needing to quiesce incoming operations (e.g., reads and writes) to the writeable data object.
Abstract:
Systems, methods, and computer program products implementing hybrid file structures for data storage are provided. One embodiment of a method performed in a computer-based storage system includes writing a file as data blocks in an array of storage devices. The method includes associating the data blocks with metadata related to at least one location in the array of storage devices for later access to the data blocks. The file is represented as a hierarchical data structure having a plurality of nodes. A first portion of nodes has a first span type, and a second portion of nodes has a second span type. The data structure includes a buftree. The first span type includes a fixed-span type. The second span type includes a variable-span type.
Abstract:
Techniques are provided for providing a storage abstraction layer for a composite aggregate architecture. A storage abstraction layer is utilized as an indirection layer between a file system and a storage environment. The storage abstraction layer obtains characteristic of a plurality of storage providers that provide access to heterogeneous types of storage of the storage environment (e.g., solid state storage, high availability storage, object storage, hard disk drive storage, etc.). The storage abstraction layer generates storage bins to manage storage of each storage provider. The storage abstraction layer generates a storage aggregate from the heterogeneous types of storage as a single storage container. The storage aggregate is exposed to the file system as the single storage container that abstracts away from the file system the management and physical storage details of data of the storage aggregate.
Abstract:
Techniques are provided for providing a storage abstraction layer for a composite aggregate architecture. A storage abstraction layer is utilized as an indirection layer between a file system and a storage environment. The storage abstraction layer obtains characteristic of a plurality of storage providers that provide access to heterogeneous types of storage of the storage environment (e.g., solid state storage, high availability storage, object storage, hard disk drive storage, etc.). The storage abstraction layer generates storage bins to manage storage of each storage provider. The storage abstraction layer generates a storage aggregate from the heterogeneous types of storage as a single storage container. The storage aggregate is exposed to the file system as the single storage container that abstracts away from the file system the management and physical storage details of data of the storage aggregate.
Abstract:
Techniques are provided for providing a storage abstraction layer for a composite aggregate architecture. A storage abstraction layer is utilized as an indirection layer between a file system and a storage environment. The storage abstraction layer obtains characteristic of a plurality of storage providers that provide access to heterogeneous types of storage of the storage environment (e.g., solid state storage, high availability storage, object storage, hard disk drive storage, etc.). The storage abstraction layer generates storage bins to manage storage of each storage provider. The storage abstraction layer generates a storage aggregate from the heterogeneous types of storage as a single storage container. The storage aggregate is exposed to the file system as the single storage container that abstracts away from the file system the management and physical storage details of data of the storage aggregate.
Abstract:
Techniques are provided for providing a storage abstraction layer for a composite aggregate architecture. A storage abstraction layer is utilized as an indirection layer between a file system and a storage environment. The storage abstraction layer obtains characteristic of a plurality of storage providers that provide access to heterogeneous types of storage of the storage environment (e.g., solid state storage, high availability storage, object storage, hard disk drive storage, etc.). The storage abstraction layer generates storage bins to manage storage of each storage provider. The storage abstraction layer generates a storage aggregate from the heterogeneous types of storage as a single storage container. The storage aggregate is exposed to the file system as the single storage container that abstracts away from the file system the management and physical storage details of data of the storage aggregate.
Abstract:
Techniques are provided for selectively storing data into allocation areas using streams. A set of allocation areas (e.g., ranges of block numbers such as virtual block numbers) are defined for a storage device. Data having particular characteristics (e.g., user data, metadata, hot data, cold data, randomly accessed data, sequentially accessed data, etc.) will be sent to the storage device for selective storage in corresponding allocation areas. For example, when a file system receives a write stream of hot data, the hot data may be assigned to a stream. The stream will be tagged using a stream identifier that is used as an indicator to the storage device to process data of the stream using an allocation area defined for hot data. In this way, data having different characteristics will be stored/confined within particular allocation areas of the storage device to reduce fragmentation and write amplification.
Abstract:
Techniques are provided for providing a storage abstraction layer for a composite aggregate architecture. A storage abstraction layer is utilized as an indirection layer between a file system and a storage environment. The storage abstraction layer obtains characteristic of a plurality of storage providers that provide access to heterogeneous types of storage of the storage environment (e.g., solid state storage, high availability storage, object storage, hard disk drive storage, etc.). The storage abstraction layer generates storage bins to manage storage of each storage provider. The storage abstraction layer generates a storage aggregate from the heterogeneous types of storage as a single storage container. The storage aggregate is exposed to the file system as the single storage container that abstracts away from the file system the management and physical storage details of data of the storage aggregate.
Abstract:
Techniques are provided for selectively storing data into allocation areas using streams. A set of allocation areas (e.g., ranges of block numbers such as virtual block numbers) are defined for a storage device. Data having particular characteristics (e.g., user data, metadata, hot data, cold data, randomly accessed data, sequentially accessed data, etc.) will be sent to the storage device for selective storage in corresponding allocation areas. For example, when a file system receives a write stream of hot data, the hot data may be assigned to a stream. The stream will be tagged using a stream identifier that is used as an indicator to the storage device to process data of the stream using an allocation area defined for hot data. In this way, data having different characteristics will be stored/confined within particular allocation areas of the storage device to reduce fragmentation and write amplification.