Abstract:
An illustrative embodiment disclosed herein is a computer-implemented method. In some embodiments, the method includes uploading, by a processor, an object to a source bucket in an object store and creating, by the processor, a lambda bucket in the object store that is symlinked to the source bucket. In some embodiments, the lambda bucket is associated with a transformation function. In some embodiments, the method includes associating, by the processor, a lambda function with the object in the source bucket, receiving, by the processor, a request to download the object from the lambda bucket, detecting, by the processor, that the object is in the source bucket and associated with the lambda function, fetching, by the processor, the object from the source bucket, generating, by the processor, a transformed object by invoking the lambda function and the transformation function on the object, and downloading, by the processor, the transformed object.
Abstract:
Communication between VMs running in a clustered virtualization environment and a distributed storage system is managed. The distributed storage system consists of multiple storage/compute nodes. A hypervisor communicates with a controller in a local node on behalf of a VM. The controller will redirect a connection to a second controller on different node. The controller to which the connection is redirected is selected so as to distribute the load evenly across the cluster.
Abstract:
Techniques for performing compression operations on persistently-stored data blocks during read/write commands. A method embodiment performs in-line data compression operations over data blocks referenced by a caller. The in-line data compression operations are performed during execution of a storage input-output (I/O) command, between the event of receipt of the storage I/O command and the event of returning status of the storage I/O command. The storage I/O operation is associated with at least one data group comprising one or more data blocks that are identified by the caller. Upon receipt of the storage I/O command, one or more compression rules are applied to the data blocks to determine one or more compression parameters, which compression parameters are used to form specific compression operations that are performed over at least a portion of the data group. The status pertaining to the execution of the storage I/O operation is returned to the caller.
Abstract:
A method for providing real time replication status for a networked virtualization environment for storage management, includes scanning metadata to identify replication status for all virtual disks (vDisks) in the networked virtualization environment, generating replication tasks for vDisks that are identified as under replicated based on the scan, performing the replication tasks, monitoring the progress of the replication tasks and determining the real time replication status of the networked virtualization environment based on the scanned metadata and the monitored progress of the replication tasks.
Abstract:
A technique allows instantiation and running on demand of long-term snapshot storage services of an archival storage system at various geographical locations. Storage service instances are configured to provide storage and retrieval of large amounts of point-in-time images or snapshots (e.g., recovery points) of application workloads stored as objects on one or more buckets of a shared object store. The storage service instances may contemporaneously serve snapshots of a same set of buckets on the shared object store without interfering with each other in a contention-free manner. That is, the technique enables storage service instances that are associated with snapshot workload data and/or metadata stored, e.g., as objects of a recovery point, on the same set of buckets to coexist without knowledge of each other. The storage service instances can be created and destroyed on-demand by splitting and merging the existing instances.
Abstract:
A unified namespace technique provides coherent access to unstructured data across different data access protocols having different logical constructs that are stored and managed on a storage system. A control plane infrastructure operates in connection with storage services to provide support for a vast array of storage platforms including file servers of a file system and object storage servers of an object store. Metadata associated with a data access transaction is processed separately and natively by a protocol stack of a particular storage service according to a particular data access protocol. The processed metadata is stored native to the access protocol in a metadata store associated with the particular storage service and is made available to the protocol stacks of the other storage services. Processed metadata is made available to the protocol stacks via an event notification logging service implemented as a message bus. A single canonical instance of the data is maintained for all of the logical constructs served by the storage system.
Abstract:
An illustrated embodiment disclosed herein relates to a hyper converged infrastructure (HCI) having an object storage system (OSS) with an apparatus having a processor with programmed instructions that when executed cause the processor to perform operations. The operations include specifying a first object in the OSS as an executable, specifying a second object in the OSS as an environment on which to run the first object, specifying a third object in the OSS as data to be processed by the first object, and constructing and storing a composite object in the OSS that associates the first, second, and third objects. The composite object provides a technical improvement, providing more direct access to compute resources.
Abstract:
Systems, methods, and computer program products for distributed data storage. A spot defragmentation method commences upon receiving an incoming storage I/O command to access a subject logical object that is composed of subject logical data blocks. The method continues by retrieving a block map that characterizes spatial relationships between the subject logical data blocks and instances of respective subject physical data blocks that store the subject logical object on a storage device. During processing of the incoming storage I/O command, the method determines occurrences of, and locations of one or more fragmented physical data blocks. A defragmentation operation is initiated to coalesce the fragmented physical data blocks to another location. The defragmentation operation is initiated before completing the storage I/O command. In some instances, defragmentation operations are performed over a snapshot file such that upon completing the defragmentation operations and/or spot coalesce operations, the snapshot file can be deleted.
Abstract:
A method for providing real time replication status for a networked virtualization environment for storage management, includes scanning metadata to identify replication status for all virtual disks (vDisks) in the networked virtualization environment, generating replication tasks for vDisks that are identified as under replicated based on the scan, performing the replication tasks, monitoring the progress of the replication tasks and determining the real time replication status of the networked virtualization environment based on the scanned metadata and the monitored progress of the replication tasks.
Abstract:
An illustrative embodiment disclosed herein is an apparatus including a processor having programmed instructions to place a first compute resource in a storage node of an object storage platform and to place a second compute resource in a compute node in a client coupled to the object storage platform via a public network. In some embodiments, unstructured data is stored in the storage node. In some embodiments, the first compute resource of the storage node preprocesses the unstructured data. In some embodiments, the preprocessed unstructured data is sent to the compute node. In some embodiments, the second compute resource trains a machine learning (ML) model using the preprocessed unstructured data.