Abstract:
Techniques are provided for processing file system requests using a super cluster of clusters of nodes. Mirror file systems for processing the requests are exported through multiple clusters in the super cluster. A cluster may be assigned to an active or passive role for processing file system requests for a set of mirror file systems. A super cluster bundle, or mapping between a cluster in the super cluster and a file system resource on the set of mirror file systems, is created to process the file system requests. The super cluster bundle represents an amount of work assigned to the cluster. A super cluster bundle is reassigned from one cluster to another in response to a failover, or in response to a load balancing determination.
Abstract:
In an embodiment, a computer stores source files and source clone files in a source filesystem. The source clone files are shallow copies of the source files, which initially share the same data blocks as the source files. A source file or a source clone file may eventually be modified, where some of the shared blocks are replaced by modified blocks. In a first phase, all data blocks of all source files that are not clones are replicated to a standby filesystem. A second phase compares each clone file on the source filesystem with its base file to detect a set of differed blocks. The second phase copies the differed blocks into the standby filesystem and, in the standby filesystem, applies the differed blocks to a corresponding clone file. Efficiency of keeping the standby filesystem synchronized with the source filesystem is improved by transferring, from the source filesystem to the standby filesystem, only one copy of only data blocks that are modified.
Abstract:
Techniques are provided for providing a fully active and non-replicated block storage solution in a clustered filesystem that implements cache coherency. In a clustered filesystem where one or more data blocks are stored in a respective cache of each host node of a plurality of host nodes, a request is received at a host node of the plurality of host nodes from a client device to write the one or more data blocks to a shared storage device. In response to the request, the one or more data blocks are stored in the cache of the host node and a particular notification is sent to another host node of the plurality of host nodes that the one or more data blocks have been written to the shared storage device. In response to receiving the notification, the other host node invalidates a cached copy of the one or more data blocks in the respective cache of the other host node.
Abstract:
Techniques are provided for exporting one or more file systems through multiple nodes of a cluster. One or more of the exported file systems may be simultaneously exported through a single node of the cluster. A bundle, or mapping, between a node in the cluster and a file system, is created. The bundle represents an amount of work assigned to the node. A bundle is reassigned from one node to another node in the cluster in response to a failover, an addition of a new node to the cluster, or in response to a load balancing determination.
Abstract:
The approaches described herein provide support for application specific policies for conventional operating systems. In an embodiment, a kernel module representing a kernel subsystem is executed within an operating system's kernel. The kernel subsystem may be configured to respond to particular requests with one or more default actions. Additionally, the kernel subsystem may define a number of sub-modules which represent application specific policies that deviate from the default actions. Each sub-module may define one or more sets of conditions which indicate when the sub-module is applicable to a request and one or more sets of corresponding actions to take when the conditions are met. When an application sends a request to the kernel subsystem, the kernel subsystem determines whether the request meets the conditions of a particular sub-module. If the particular sub-module's conditions are met, the kernel subsystem performs the corresponding actions of the particular sub-module.
Abstract:
In an embodiment, a computer stores source files and source clone files in a source filesystem. The source clone files are shallow copies of the source files, which initially share the same data blocks as the source files. A source file or a source clone file may eventually be modified, where some of the shared blocks are replaced by modified blocks. In a first phase, all data blocks of all source files that are not clones are replicated to a standby filesystem. A second phase compares each clone file on the source filesystem with its base file to detect a set of differed blocks. The second phase copies the differed blocks into the standby filesystem and, in the standby filesystem, applies the differed blocks to a corresponding clone file. Efficiency of keeping the standby filesystem synchronized with the source filesystem is improved by transferring, from the source filesystem to the standby filesystem, only one copy of only data blocks that are modified
Abstract:
A heatmap is used to identify access patterns of ranges of data blocks at specific times, in order to optimize the size of the set of replicate nodes, for the purpose of reducing access latency. In an embodiment, the heatmap is used to enforce minimum replication of each data block such as when a replicate node crashes and is replaced. In an embodiment, the heatmap is used to adjust the minimum replication of a data block such as during and after a demand spike. In an embodiment, each data block is replicated on a minimum amount of respective replicate nodes of a cluster. A first node requests access to data blocks. Based on the requesting the access, a heatmap is modified, and the data blocks are replicated to the first node. Based on the heatmap, the minimum amount of nodes in the respective replicate nodes for at least one data block is adjusted.
Abstract:
Techniques are provided for providing a fully active and non-replicated block storage solution in a clustered filesystem that implements cache coherency. In a clustered filesystem where one or more data blocks are stored in a respective cache of each host node of a plurality of host nodes, a request is received at a host node of the plurality of host nodes from a client device to write the one or more data blocks to a shared storage device. In response to the request, the one or more data blocks are stored in the cache of the host node and a particular notification is sent to another host node of the plurality of host nodes that the one or more data blocks have been written to the shared storage device. In response to receiving the notification, the other host node invalidates a cached copy of the one or more data blocks in the respective cache of the other host node.
Abstract:
Techniques are provided for exporting one or more file systems through multiple nodes of a cluster. One or more of the exported file systems may be simultaneously exported through a single node of the cluster. A bundle, or mapping, between a node in the cluster and a file system, is created. The bundle represents an amount of work assigned to the node. A bundle is reassigned from one node to another node in the cluster in response to a failover, an addition of a new node to the cluster, or in response to a load balancing determination.
Abstract:
Techniques are provided for efficiently managing multi-tier storage systems with multiple types of storage class tiers encompassed in a single tiered range volume. In an embodiment, a storage manager creates a first virtual volume, where the storage manager exposes to a first tiered range volume client a first virtual address range for the first virtual volume that represents logical addresses of data blocks within storage portions from at least two storage class tiers of a set of multiple storage class tiers. Each storage portion represents an allocated range from a storage class tier. A storage class tier represents a set of multiple storage devices. The storage manager maintains a storage map of storage portions from multiple storage class tiers to the first virtual address range within the first virtual volume which includes mapping a first virtual address to a first logical address within a first storage portion from a first storage class tier and mapping a second virtual address to a second logical address within a second storage portion from a second storage class tier. A tiered range volume client of the first virtual volume stores a first data block at a first address within the first storage portion. Then the storage manager adds a third storage portion from a third storage class tier to the first virtual volume that maps to a third allocated range, where the third allocated range of the first virtual volume covers at least a portion of the first allocated range, including the first address where a first data block is stored. When adding a third storage portion, the storage manager modifies the storage map to map the first virtual address for the first data block to a location within the third storage portion. The storage manager then moves the first data block to the third storage portion within the third storage class tier that corresponds to a location corresponding to the first address within the third storage portion.