Abstract:
A method for providing real time replication status for a networked virtualization environment for storage management, includes scanning metadata to identify replication status for all virtual disks (vDisks) in the networked virtualization environment, generating replication tasks for vDisks that are identified as under replicated based on the scan, performing the replication tasks, monitoring the progress of the replication tasks and determining the real time replication status of the networked virtualization environment based on the scanned metadata and the monitored progress of the replication tasks.
Abstract:
Systems and methods for scheduling storage management tasks over predicted user tasks in a distributed storage system. A method commences upon receiving a set of historical stimulus records that characterize management tasks that are run in the storage system. A corresponding set of historical response records comprising system metrics associated with execution of the system tasks is also received. A learning model is formed from the stimulus records and the response records and formatted to be used as a predictor. A set of forecasted user tasks is input as new stimulus records to the predictor to determine a set of forecasted system metrics that would result from running the forecasted user tasks. Management tasks are selected so as not to impact the forecasted user tasks. Management tasks can be selected based on non-contentions resource usage between historical management task resource usage and predictions of resource usage by the user tasks.
Abstract:
Techniques for performing compression operations on persistently-stored data blocks during read/write commands. A method embodiment performs in-line data compression operations over data blocks referenced by a caller. The in-line data compression operations are performed during execution of a storage input-output (I/O) command, between the event of receipt of the storage I/O command and the event of returning status of the storage I/O command. The storage I/O operation is associated with at least one data group comprising one or more data blocks that are identified by the caller. Upon receipt of the storage I/O command, one or more compression rules are applied to the data blocks to determine one or more compression parameters, which compression parameters are used to form specific compression operations that are performed over at least a portion of the data group. The status pertaining to the execution of the storage I/O operation is returned to the caller.
Abstract:
A method for providing real time replication status for a networked virtualization environment for storage management, includes scanning metadata to identify replication status for all virtual disks (vDisks) in the networked virtualization environment, generating replication tasks for vDisks that are identified as under replicated based on the scan, performing the replication tasks, monitoring the progress of the replication tasks and determining the real time replication status of the networked virtualization environment based on the scanned metadata and the monitored progress of the replication tasks.
Abstract:
Techniques for performing compression operations on persistently-stored data blocks during read/write commands. A method embodiment performs in-line data compression operations over data blocks referenced by a caller. The in-line data compression operations are performed during execution of a storage input-output (I/O) command, between the event of receipt of the storage I/O command and the event of returning status of the storage I/O command. The storage I/O operation is associated with at least one data group comprising one or more data blocks that are identified by the caller. Upon receipt of the storage I/O command, one or more compression rules are applied to the data blocks to determine one or more compression parameters, which compression parameters are used to form specific compression operations that are performed over at least a portion of the data group. The status pertaining to the execution of the storage I/O operation is returned to the caller.
Abstract:
Systems and methods for scheduling storage management tasks over predicted user tasks in a distributed storage system. A method commences upon receiving a set of historical stimulus records that characterize management tasks that are run in the storage system. A corresponding set of historical response records comprising system metrics associated with execution of the system tasks is also received. A learning model is formed from the stimulus records and the response records and formatted to be used as a predictor. A set of forecasted user tasks is input as new stimulus records to the predictor to determine a set of forecasted system metrics that would result from running the forecasted user tasks. Management tasks are selected so as not to impact the forecasted user tasks. Management tasks can be selected based on non-contentions resource usage between historical management task resource usage and predictions of resource usage by the user tasks.
Abstract:
Techniques for performing compression operations on persistently-stored data blocks during read/write commands. A method embodiment performs in-line data compression operations over data blocks referenced by a caller. The in-line data compression operations are performed during execution of a storage input-output (I/O) command, between the event of receipt of the storage I/O command and the event of returning status of the storage I/O command. The storage I/O operation is associated with at least one data group comprising one or more data blocks that are identified by the caller. Upon receipt of the storage I/O command, one or more compression rules are applied to the data blocks to determine one or more compression parameters, which compression parameters are used to form specific compression operations that are performed over at least a portion of the data group. The status pertaining to the execution of the storage I/O operation is returned to the caller.
Abstract:
Dynamic erasure coding for computing and data storage systems. A method embodiment commences upon accessing a set of fault tolerance policy attributes associated with the computing and data storage system. The topology of the system is analyzed to form mappings between the computing nodes of the system and the availability domains of the system. Based on the fault tolerance policy attributes, the topology, and the generated mapping, a plurality of feasible erasure coding configurations are generated. The feasible erasure coding configurations are scored. One or more high-scoring feasible erasure coding configurations are selected and deployed to the computing and data storage system. The method is repeated when there is a change in the fault tolerance policy attributes or in the topology. Depending on the topology and/or the nature of a change in the topology, more than one erasure coding configurations can be deployed onto the computing and data storage system.
Abstract:
Systems and methods for scheduling storage management tasks over predicted user tasks in a distributed storage system. A method commences upon receiving a set of historical stimulus records that characterize management tasks that are run in the storage system. A corresponding set of historical response records comprising system metrics associated with execution of the system tasks is also received. A learning model is formed from the stimulus records and the response records and formatted to be used as a predictor. A set of forecasted user tasks is input as new stimulus records to the predictor to determine a set of forecasted system metrics that would result from running the forecasted user tasks. Management tasks are selected so as not to impact the forecasted user tasks. Management tasks can be selected based on non-contentions resource usage between historical management task resource usage and predictions of resource usage by the user tasks.
Abstract:
Systems for self-configuring health monitoring instrumentation for clustered storage platforms. Master and slave health modules implement a health monitoring system in a clustered virtualization environment comprising a plurality of nodes of the cluster with an installed health module instance running on the nodes. The health module system may gather and analyze data on a node level and at a cluster level to manage the cluster. The cluster health module system observes I/O commands issued to, and I/O command responses returned from, a common storage pool. Health data is stored in the storage pool.