Abstract:
Systems and methods for efficiently using solid-state devices are provided. Some embodiments provide for a data processing system that uses a non-volatile solid-state device as a circular log, with the goal of aligning data access patterns to the underlying, hidden device implementation, in order to maximize performance. In addition, metadata can be interspersed with data in order to align data access patterns to the underlying device implementation. Multiple input/output (I/O) buffers can also be used to pipeline insertions of metadata and data into a linear log. The observed queuing behavior of the multiple I/O buffers can be used to determine when the utilization of the storage device is approaching saturation (e.g., in order to predict excessively-long response times). Then, the I/O load on the storage device may be shed when utilization approaches saturation. As a result, the overall response time of the system is improved.
Abstract:
Described herein is a high-availability storage system having hierarchical levels of storage functions. The storage system may comprise one or more hierarchical levels, each hierarchical level comprising physical servers and be assigned to perform a particular set of storage functions. Each physical server may implement one or more VMs configured to perform only the set of storage functions assigned to the hierarchical level on which the VM executes. VMs of a first hierarchical level may be configured to organize the VMs of a second hierarchical level into a redundant array of storage access servers for providing data reliability and high-availability of the storage system. VMs of a first hierarchical level are configured to produce and route sub-requests to the VMs of a second hierarchical level. Failure of a sub-request is detected and remedied by a VM of the first hierarchical level.
Abstract:
Systems and methods for efficiently using solid-state devices are provided. Some embodiments provide for a data processing system that uses a non-volatile solid-state device as a circular log, with the goal of aligning data access patterns to the underlying, hidden device implementation, in order to maximize performance. In addition, metadata can be interspersed with data in order to align data access patterns to the underlying device implementation. Multiple input/output (I/O) buffers can also be used to pipeline insertions of metadata and data into a linear log. The observed queuing behavior of the multiple I/O buffers can be used to determine when the utilization of the storage device is approaching saturation (e.g., in order to predict excessively-long response times). Then, the I/O load on the storage device may be shed when utilization approaches saturation. As a result, the overall response time of the system is improved.
Abstract:
A persistent cache is implemented in a flash memory that includes a journal section that stores metadata and a low frequency section and a high frequency section that store data entries. Writing new metadata to the persistent cache includes sequentially advancing to a next sector containing an invalid metadata entry, saving a working copy of the sector in RAM, writing metadata corresponding to one or more new data entries in the working copy, and overwriting the sector in the flash memory containing the invalid entry with the working copy. Writes to the low frequency and high frequency sections occur sequentially in the current locations of a low frequency section pointer and a high frequency section pointer, respectively. In a persistent cache, the reconstruction of a non-persistent cache utilizes the metadata entry that has the most recent timestamp.
Abstract:
Described herein is a storage system having hierarchical levels of storage functions. The storage system may comprise one or more hierarchical levels, each hierarchical level comprising a server pool of one or more physical servers. Each hierarchical level may be assigned to perform a particular set of storage functions. Each physical server may implement one or more VMs, each VM executing a storage operating system that provides an overall set of storage functions. Each VM is configured to perform only the set of storage functions assigned to the hierarchical level on which the VM executes. The set of storage functions assigned to a VM may comprise a sub-set of the overall storage functions of the storage operating system. In some embodiments, VMs of a first hierarchical level provide storage transaction functions and VMs of a second hierarchical level provide storage access functions.
Abstract:
One or more techniques and/or systems are provided for performing host side deduplication. Host side deduplication may be performed upon writeable data within a write request received at a host computing device configured to access data stored by a storage server. The host side deduplication may be performed at the host computing device to determine whether the writeable data is already stored by the storage server based upon querying a host side cache comprising data stored by a storage server and/or a data structure comprising unique signatures of data stored by the storage server. If the writeable data is stored by the storage server, then a deduplication notification excluding the writeable data may be sent to the storage server, otherwise a write command comprising the writeable data may be sent. Accordingly, unnecessary network traffic of redundant data already stored by the storage server may be reduced.