Abstract:
Technology is disclosed for a data storage architecture for providing enhanced storage resiliency for a data object. The data storage architecture can be implemented in a single-tier configuration and/or a multi-tier configuration. In the single-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data fragments, which are stored across many storage devices. In the multi-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data segments, which are sent to one or more tiers of storage nodes. Each of the storage nodes further encodes the data segment to generate many data fragments representing the data segment, which are stored across many storage devices associated with the storage node. The I/O operations for rebuilding the data in case of device failures is spread across many storage devices, which minimizes the wear of a given storage device.
Abstract:
Methods and systems that permit file protocols and object protocols to co-exist using a file namespace and an object namespace in a unified storage system are disclosed. Data stored in the unified storage system are file-objects that preserve the behaviors expected by both file clients and object clients.
Abstract:
Methods, non-transitory computer readable media, and devices that more effectively manage snapshots by creating a namespace including files described by stored file recipe objects and each comprising fragments, directories described by metadata in the file recipe objects and each comprising one or more of the files, and snapshots described by stored snapshot request objects. Content of one of the directories are identified based on an identification of corresponding ones of the file recipe objects that existed within the namespace at a specified time. At least one of the files, included in the identified content and existing within the namespace at the specified time, is accessed in response to one or more received requests. A garbage collection operation is periodically performed to delete the recipe objects that are marked for deletion by tombstone objects and are unreferenced by any of the snapshots as determined based on the snapshot request objects.
Abstract:
Techniques are provided for offloading the management of sensor data and generating custom views of sensor data. Sensor data received from a data network through a message is stored within storage managed by a computing device. A handle is generated to identify the sensor data. The sensor data within the message is replaced with the handle, and the message is transmitted to a device within the data network. The device may use handles of sensor data to request custom views of sensor data.
Abstract:
Methods, non-transitory computer readable media, and devices that more effectively manage snapshots by creating a namespace including files described by stored file recipe objects and each comprising fragments, directories described by metadata in the file recipe objects and each comprising one or more of the files, and snapshots described by stored snapshot request objects. Content of one of the directories are identified based on an identification of corresponding ones of the file recipe objects that existed within the namespace at a specified time. At least one of the files, included in the identified content and existing within the namespace at the specified time, is accessed in response to one or more received requests. A garbage collection operation is periodically performed to delete the recipe objects that are marked for deletion by tombstone objects and are unreferenced by any of the snapshots as determined based on the snapshot request objects.
Abstract:
Methods and systems for executing code are provided. A message with executable code is received by a processor that utilizes the message to generate a key associated with a value having executable code stored at a key-value store. The processor retrieves the value from the key-value store using the generated key. The processor then merges the executable code in the received message and the executable code in the retrieved value to generate a merged value for code execution.
Abstract:
Technology is disclosed for a data storage architecture for providing enhanced storage resiliency for a data object. The data storage architecture can be implemented in a single-tier configuration and/or a multi-tier configuration. In the single-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data fragments, which are stored across many storage devices. In the multi-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data segments, which are sent to one or more tiers of storage nodes and at least one latent storage. Each of the storage nodes further encodes the data segment to generate many data fragments representing the data segment, which are stored across many storage devices associated with the storage node. The I/O operations for rebuilding the data in case of device failures is spread across many storage devices, which minimizes the wear of a given storage device.
Abstract:
Technology is disclosed for a data storage architecture for providing enhanced storage resiliency for a data object. The data storage architecture can be implemented in a single-tier configuration and/or a multi-tier configuration. In the single-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data fragments, which are stored across many storage devices. In the multi-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data segments, which are sent to one or more tiers of storage nodes and at least one latent storage. Each of the storage nodes further encodes the data segment to generate many data fragments representing the data segment, which are stored across many storage devices associated with the storage node. The I/O operations for rebuilding the data in case of device failures is spread across many storage devices, which minimizes the wear of a given storage device.
Abstract:
Techniques for operating a storage front-end system are disclosed. The techniques include identifying a synchronous group of data storage devices across two or more enclosures, each of which comprise a plurality of data storage devices. Data across the data storage devices is accessible by a storage front-end system as an aggregate memory space. The techniques further include sending an activation request to the enclosures to synchronously activate the data storage devices in the synchronous group and performing a data maintenance task in the aggregate memory space of the data storage devices.
Abstract:
Systems and methods that allow operators to configure how files and directories are placed within file system views into a storage system, and how these configured file placements are performed by the distributed system. Possible features include a mechanism by which the desired placement of files and directories can be specified, a mechanism by which the placement goals can be realized in a given topology of a distributed system, and a mechanism by which changes to the placement of objects can applied retroactively. The disclosed embodiments may be able to scale to managing hundreds of billions of files spanning thousands of file system views, especially in the presence of disconnected operation.