Abstract:
The present disclosure provides persistent storage for a master copy using operation numbers. A master copy can include a B-tree with references to corresponding data. When provisioning a slave copy, the master copy sends a point-in-time copy of the B-tree to the slave copy, which stores a copy of the B-tree, allocates the necessary space, and updates the references of the B-tree to point to a local storage before the data is transferred. When writing the data to persistent storage, a snapshot created on the master copy is an operation that is replicated to the slave copy. The snapshot is generated using a volume view that includes changes to chunks of data of the master copy since a previous snapshot, as determined using the operation number for the previous snapshot. Data (and metadata) for the snapshot is written to persistent storage while new I/O operations are processed.
Abstract:
The current document describes a storage system that provides a storage volume with a block-data interface to one or more clients. The storage system provides a key-value store that is associated with the storage volume. The clients access the key-value store via a key-value interface that provides a number of key-value operations to read, write, list, and delete key-value records from the key-value store. In some implementations, the key-value interface provides additional key-value operations that support the use of the key-value store by more than one client at a time. An environment in which the key-value store is used by the clients to retain I/O throttling parameters is described.
Abstract:
A slave storage is provisioned using metadata of a master B-tree and updates to references (e.g., offsets) pertaining to data operations of the master B-tree. Master-slave pairs can be used to provide data redundancy, and a master copy can include the master B-tree with references to corresponding data. When provisioning a slave copy, the master sends a B-tree copy to the slave, which stores the slave B-tree copy, allocates the necessary space on local storage, and updates respective offsets of the slave B-tree copy to point to the local storage. Data from the master can then be transferred to the slave and stored according to a note and commit process that ensures operational sequence of the data. Operations received to the master during the process can be committed to the slave copy until the slave is consistent with the master and able to take over as master in the event of a failure.
Abstract:
Techniques are described for formally expressing whether sequences of operations performed on block storage devices are sequential or random. In embodiments, determinations of whether these sequences of operations are sequential or random may be used to predict latencies involved with running particular workloads, and to predict representative workloads for particular latencies.
Abstract:
A block-based storage system may implement dynamic durability adjustment for page cache write logging. A rate of incoming write requests for data volumes maintained at a storage node may be monitored. Based, at least in part, on the rate of incoming write requests, a dynamic modification to a durability property for a data volume may be made, such as enabling page cache write logging the data volume or disabling write logging for the data volume. When incoming write requests are received, a determination may be made as to whether page cache write logging for a particular data volume is enabled. For write requests with disabled page cache write logging, the page cache may be updated and the write request may be acknowledged without storing a log record describing the update in a page cache write log.
Abstract:
A block-based storage system may implement dynamic durability adjustment for page cache write logging. A rate of incoming write requests for data volumes maintained at a storage node may be monitored. Based, at least in part, on the rate of incoming write requests, a dynamic modification to a durability property for a data volume may be made, such as enabling page cache write logging the data volume or disabling write logging for the data volume. When incoming write requests are received, a determination may be made as to whether page cache write logging for a particular data volume is enabled. For write requests with disabled page cache write logging, the page cache may be updated and the write request may be acknowledged without storing a log record describing the update in a page cache write log.
Abstract:
At a first client-side component of a storage service, a metric of a storage workload directed from the first client-side component towards server components of the storage service is determined. The metric is transmitted to a particular server component, which is configured to propagate the metric to a different client-side component of the storage service. The first client-side component receives, from a server component, a different metric of a storage workload of a second client-side component. The first client-side component reschedules at least one storage request based at least in part on the different metric.
Abstract:
Persistent storage for a master copy is provided using operation numbers. A master copy can include a persistent key-value store such as a B-tree with references to corresponding data. When provisioning a slave copy, the master copy sends a point-in-time copy of the B-tree to the slave copy, which stores a copy of the B-tree, allocates the necessary space, and updates the references of the B-tree to point to a local storage before the data is transferred. When writing the data to persistent storage, a snapshot created on the master copy is an operation that is replicated to the slave copy. The snapshot is generated using a volume view that includes changes to chunks of data of the master copy since a previous snapshot, as determined using the operation number for the previous snapshot. Data (and metadata) for the snapshot is written to persistent storage while new input/output operations are processed.
Abstract:
The estimated rate of work requests expected during a time period at a first block storage device, implemented at a particular server of a storage service, exceeds a provisioned rate of the first device. At a client-side component of the storage service, a different storage server is identified, at which the rate of work requests directed during the time period to a second block storage device is anticipated to be less than the provisioned rate of the second device. At least one admission control parameter of the first device is modified to enable the first storage server to accept work requests at a rate that exceeds the provisioned rate of the first device.
Abstract:
Systems and methods are described providing detailed input/output (I/O) metric information that is collected and gathered by an agent of the storage volume. An I/O request is received by a storage volume, and the agent associates primary and secondary identifiers with that I/O request. For example, a trace may be associated with that I/O request. The agent may store this I/O metric information in a ring buffer. Further, after collection and aggregation, statistics may be published by an I/O metric service that further processes the data provided by the agent. Advantageously, interdependent relationships associated with the I/O request or I/O operations of that request may be included in those statistics. This may allow an operator to evaluate the performance of I/O requests for a network.