摘要:
A system and method for generating and storing at least one snapshot update. The system and method comprising determining a first set of blocks including blocks that have changed since a time of a previous snapshot, determining a second set of blocks based at least in part on a predetermined amount, generating at least one snapshot update of a volume based at least in part on a combination of the first set of blocks and the second set of blocks, and storing the at least one snapshot update.
摘要:
A block-level data storage system receives a request to delete a data storage volume. As a result, the data storage volume is deleted and the areas comprising the volume are released and reaped. The areas may contain non-zero data within a plurality of data storage chunks that comprise the areas. An area cleaner is configured to zero out the areas for allocation to a newly created data storage volume.
摘要:
Embodiments of system and method for performing object-modifying commands in an unstructured storage service are described. Embodiments may include utilizing one or more nodes to implement an unstructured storage service including a service interface configured to service data storage requests and data retrieval requests from remote clients. The unstructured storage service is configured to isolate access to each of multiple storage domains of the service to different clients. Embodiments may also include receiving from a given remote client, a request to store a data object to a given storage domain of the service. The request further specifying a command to modify source data for the object prior to storing the object. Embodiments may also include using the unstructured storage service to perform the command without exporting the source data from the unstructured storage service to the given remote client prior to completion of the command.
摘要:
A system and method for using a dead letter queue to regulate host computing systems performing transformation or migration operations. The system and method including reading a message from a queue, the message including one or more references to a set of data objects, with the set of data objects including one or more data objects selected for a transformation operation according to a transformation scheme. The system and method further including performing a mitigation action upon determination that the message is one of a number of unsuccessfully processed messages in the queue that are associated with the computer system, when the number of unsuccessfully processed messages exceed a threshold.
摘要:
Write optimization for block-based storage performing snapshot operations may be implemented. Write requests for a particular data volume may be received for which a snapshot operation is in progress. A determination may be made as to whether a data chunk of the data volume modified as part of the write request has not yet been stored to a remote snapshot data store as part of the snapshot operation. For a data chunk that is to be modified and that has not yet been stored, the data chunk may be stored in a local in-memory volume snapshot buffer. Once the data chunk is stored in the in-memory volume snapshot buffer, the write request may be performed and acknowledged as complete. The data chunk may be sent to the remote snapshot data store asynchronously with regard to the acknowledgment of the write request.
摘要:
Embodiments of system and method for performing object-modifying commands in an unstructured storage service are described. Embodiments may include utilizing one or more nodes to implement an unstructured storage service including a service interface configured to service data storage requests and data retrieval requests from remote clients. The unstructured storage service is configured to isolate access to each of multiple storage domains of the service to different clients. Embodiments may also include receiving from a given remote client, a request to store a data object to a given storage domain of the service. The request further specifying a command to modify source data for the object prior to storing the object. Embodiments may also include using the unstructured storage service to perform the command without exporting the source data from the unstructured storage service to the given remote client prior to completion of the command.
摘要:
A system and method for maintaining data integrity during data transformation operations. The system and method include obtaining a message from a set of queues, obtaining, from a first data store, a data object indicated by the message, and generating, at a first host, a set of error detection codes corresponding to a transformation of the data object according to a transformation scheme. The system and method further include, transforming, at a second host different from the first host, the data object according to the transformation scheme into the transformation of the data object, verifying the transformation against the set of error detection codes, and storing the transformation in a second data store.
摘要:
A block-based storage system may implement efficient replication for restoring a data volume from a reduced durability state. A storage node that is not replicating write requests for a data volume may determine that replication for the data volume is to be enabled. A peer storage node may be identified that maintains a stale replica of the data volume. One or more replication operations may be performed to update stale data chunks in the stale replica of the data volume with current data chunks without updating data chunks in the stale replica of the data volume that are current. Stale replicas that are no longer needed may be deleted according timeouts or the amount of stale data in the replica.
摘要:
A block-based storage system may implement reducing durability state for a data volume. A determination may be made that storage node replicating write requests for a data volume is unavailable. In response, subsequent write requests may be processed according to a reduced durability state for the data volume such that replication for the data volume may be disabled for the storage node. Write requests may then be completed at a fewer number of storage nodes prior to acknowledging the write request as complete. Durability state for the data volume may be increase in various embodiments. A storage node may be identified and replication operations may be performed to synchronize the current data volume at the storage node with a replica of the data volume maintained at the identified storage node.
摘要:
Methods and systems for cursor remirroring are disclosed. A mirroring process is initiated for a plurality of chunks stored by a master node. The mirroring process comprises visiting a sequence of one or more of the chunks and, for at least some of the chunks, copying chunk data or metadata to a slave node. During the initiated mirroring process, a request is received for a write operation on one of the chunks stored by the master node. If the chunk in the request has been visited in the mirroring process, the write operation is performed on the master node and on the slave node. If the chunk in the request has not been visited, the write operation is performed on the master node and postponed on the slave node until the chunk in the request has been visited in the mirroring process.