Abstract:
Systems and techniques are described for transferring data. A described technique includes determining that a first replica of an object stored at a first host has become available to a distributed storage system after previously being unavailable to the distributed storage system. The object includes a range of memory addresses at which data of the object is stored. In response to determining that the first replica has become available, resyncing data for the first replica is obtained. The resyncing data indicates whether each range of memory addresses is synchronized at the first replica with other replicas of the object. Tracking data for the first replica is obtained. The tracking data indicates whether data stored at the range of memory addresses of the object has been modified at a second replica while the first replica was unavailable. The resyncing data is updated based on the tracking data.
Abstract:
Certain embodiments described herein are generally directed to generating an updated version of an existing code block in response to a data update to an existing data block of a plurality of data blocks in a distributed multi-mirror erasure coding system. For example, in some embodiments, a first operation cost to generate the updated version of the existing code block by a first method based on the plurality of data blocks and the data update is determined. In certain embodiments, a second operation cost to generate the updated version of the existing code block by a second method based on the existing code block, the existing data block, and the data update is determined. In some embodiments, if the first operation cost is less than or equal to the second operation cost, the first method is used. Otherwise, the second method is used.
Abstract:
Systems and techniques are described for transferring data. A described technique includes determining that a first replica of an object stored at a first host has become available to a distributed storage system after previously being unavailable to the distributed storage system. The object includes a range of memory addresses at which data of the object is stored. In response to determining that the first replica has become available, resyncing data for the first replica is obtained. The resyncing data indicates whether each range of memory addresses is synchronized at the first replica with other replicas of the object. Tracking data for the first replica is obtained. The tracking data indicates whether data stored at the range of memory addresses of the object has been modified at a second replica while the first replica was unavailable. The resyncing data is updated based on the tracking data.
Abstract:
Systems and techniques are described for transferring data. A described technique includes receiving a request to transmit a data block from a first data storage device to a second data storage device. An attempt to read the data block from the first data storage device is made. A media error resulting from the attempt to read the data block from the first data storage device is detected. In response to detecting the media error, a new data block is generated and includes mismatched checksum data that causes a checksum mismatched error when the new data block is accessed. The new data block is transmitted for storage at the second data storage device in place of the data block.
Abstract:
An example method is provided to perform policy-based data placement in a virtualized computing environment. The method may comprise receiving a request to create a virtual disk for a virtual machine in the virtualized computing environment. The request may include a usage type associated with the virtual disk. The method may further comprise, based on a data placement policy associated with the usage type, determining a datastore on which to place the virtual disk and to store data of the usage type associated with the virtual disk.
Abstract:
A method of retransmitting a multicast message through a unicast channel in a virtual distributed storage system comprising a cluster of nodes that includes a master node and a set of agent nodes is provided. Each node maintains a copy of a directory of the virtual distributed storage system. The master node multicasts messages to the agent node through a multicast channel to update the directory. The method determines that a particular message to update the directory exceeds a maximum size limit for transmit to the agent nodes through the multicast channel. The method multicasts a retransmission trigger message from the master node to each agent node. The method receives a retransmission request message at the master node from each of the agent nodes through a unicast channel. The method sends the particular message to update the directory from the master node to each agent node through the unicast channel.