Abstract:
A distributed storage system replicates data for a primary logical storage object on a primary node of the storage system to a secondary logical storage object on a secondary node on the distributed storage system. Failures in writing data to the primary logical storage object or failures in the replication of the data to the secondary logical storage object can cause data that should be synchronized to become divergent. In cases where the data may be divergent, reconciliation operations can be performed to resynchronize the data.
Abstract:
Technology is disclosed for a data storage architecture for providing enhanced storage resiliency for a data object. The data storage architecture can be implemented in a single-tier configuration and/or a multi-tier configuration. In the single-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data fragments, which are stored across many storage devices. In the multi-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data segments, which are sent to one or more tiers of storage nodes. Each of the storage nodes further encodes the data segment to generate many data fragments representing the data segment, which are stored across many storage devices associated with the storage node. The I/O operations for rebuilding the data in case of device failures is spread across many storage devices, which minimizes the wear of a given storage device.
Abstract:
Technology is disclosed for a data storage architecture for providing enhanced storage resiliency for a data object. The data storage architecture can be implemented in a single-tier configuration and/or a multi-tier configuration. In the single-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data fragments, which are stored across many storage devices. In the multi-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data segments, which are sent to one or more tiers of storage nodes. Each of the storage nodes further encodes the data segment to generate many data fragments representing the data segment, which are stored across many storage devices associated with the storage node. The I/O operations for rebuilding the data in case of device failures is spread across many storage devices, which minimizes the wear of a given storage device.
Abstract:
Embodiments described herein provide an object store that efficiently manages and services objects for use by clients of a distributed data processing system. Illustratively, the object store may be embodied as a quasi-shared storage system that interacts with nodes of the distributed data processing system to service the objects as blocks of data stored on a plurality of storage devices, such as disks, of the storage system. To that end, an architecture of the object store may include an on-disk layout, e.g., of the storage system, and an incore layout, e.g., of the nodes, that cooperate to illustratively convert the blocks to objects for access by the clients.
Abstract:
Described herein are systems and methods for providing data policy management over application objects in a storage system environment. An application object may comprise non-virtual or virtual objects (e.g., non-virtual-based applications, virtual-based applications, or virtual storage components). An application object manager may represent application objects by producing mapping graphs and/or application object data that represent application objects in a standardized manner. A mapping graph for an application object may describe a mapping between the application object and its underlying storage objects on a storage system. Application object data may describe a mapping graph in a standardized format. Application object data representing application objects may be received by an application policy manager that manages data policies on the application objects (including virtual applications and virtual storage components) based on the received application object data. Data policies may include policies for backup, service level objectives, recovery, monitoring and/or reporting.
Abstract:
A system and method for efficiently restoring one or more data containers is provided. A common persistent consistency point image (PCPI) is identified between a source and a destination storage systems prior to the destination storage system performing a rollback operation to the commonly identified PCPI. Differential data is then transmitted from the source storage system in a line efficient manner to the destination storage system.
Abstract:
A namespace and storage management (NSM) application includes an infrastructure configured to enable efficient management of resources in a storage system environment. The NSM application executes on a NSM console and interacts with an NSM server to integrate namespace management and storage management in the storage system environment. The NSM server, in turn, interacts with one or more remote agents installed on host machines in the environment to convey application programming interface (API) function calls that enable remote management of the resources.
Abstract:
Example embodiments provide various techniques for distributing connections within a connectional parallelism architecture. In one embodiment, a method is provided where resource utilizations of connection groups are measured. Here, each connection group is assigned to one of multiple processors. A probability distribution is accessed that maps probabilities assigned to relative resource utilizations. A relative resource utilization of one of the connection groups is determined based on a resource utilization of the one connection group relative to other resource utilizations of other connection groups. A probability from the probability distribution is identified based on the determined relative resource utilization, and based on the identified probability, a connection is assigned to this connection group for execution by one of the processors assigned to this connection group.
Abstract:
Systems and methods for centralizing database manipulation for a plurality of heterogeneous databases are disclosed. A single or limited number of central servers can be used to manage a plurality of hosted client systems. With such a technique, database consistent backups can be performed without requiring altering of the central server, even when different database engines are used across the hosted client systems.
Abstract:
In one embodiment, a node coupled to one or more storage devices executes a storage input/output (I/O) stack having a volume layer. The volume layer manages volume metadata embodied as mappings from offsets of a logical unit (LUN) to extent keys associated with storage locations for extents on the one or more storage devices. Volume metadata is maintained as a dense tree metadata structure representing successive points in time. The dense tree metadata structure has multiple levels, wherein a top level of the dense tree metadata structure represents newer volume metadata changes and descending levels of the dense tree metadata structure represent older volume metadata changes. The node accesses a latest version of changes to the volume metadata by searching from the top level to the descending levels in the dense tree metadata structure.