Abstract:
A method for a transactional commit in a storage unit is provided. The method includes receiving a logical record from a storage node into a transaction engine of a storage unit of the storage node and writing the logical record into a data structure of the transaction engine. The method includes writing, to a command queue of the transaction engine, an indication to perform an atomic update using the logical record and transferring each portion of the logical record from the data structure of the transaction engine to non-persistent memory of the storage unit as a committed transaction. A storage unit for a storage system is also provided.
Abstract:
Data transformation offloading in an artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (‘GPU’) servers, including: storing, within the storage system, a dataset; identifying, in dependence upon one or more machine learning models to be executed on the GPU servers, one or more transformations to apply to the dataset; and generating, by the storage system in dependence upon the one or more transformations, a transformed dataset.
Abstract:
A non-volatile solid-state storage is provided. The non-volatile solid state storage includes a non-volatile random access memory (NVRAM) addressable by a processor external to the non-volatile solid state storage. The NVRAM is configured to store user data and metadata relating to the user data. The non-volatile solid state storage includes a flash memory addressable by the processor. The flash memory is configured to store the user data responsive to the processor directing transfer of the user data from the NVRAM to the flash memory.
Abstract:
A method for updates in a storage system is provided. The method includes writing identifiers, associated with data to be stored, to storage units of the storage system and writing trim records indicative of identifiers that are allowed to not exist in the storage system to the storage units. The method includes determining whether stored data corresponding to records of identifiers is valid based on the records of the identifiers and the trim records.
Abstract:
Data transformation caching in an artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (‘GPU’) servers, including: identifying, in dependence upon one or more machine learning models to be executed on the GPU servers, one or more transformations to apply to a dataset; generating, in dependence upon the one or more transformations, a transformed dataset; storing, within one or more of the storage systems, the transformed dataset; receiving a plurality of requests to transmit the transformed dataset to one or more of the GPU servers; and responsive to each request, transmitting, from the one or more storage systems to the one or more GPU servers without re-performing the one or more transformations on the dataset, the transformed dataset.
Abstract:
A method for a transactional commit in a storage unit is provided. The method includes receiving a logical record from a storage node into a transaction engine of a storage unit of the storage node and writing the logical record into a data structure of the transaction engine. The method includes writing, to a command queue of the transaction engine, an indication to perform an atomic update using the logical record and transferring each portion of the logical record from the data structure of the transaction engine to non-persistent memory of the storage unit as a committed transaction. A storage unit for a storage system is also provided.
Abstract:
A method of applying an address space to data storage in a non-volatile solid-state storage is provided. The method includes receiving a plurality of portions of user data for storage in the non-volatile solid-state storage and assigning to each successive one of the plurality of portions of user data one of a plurality of sequential, nonrepeating addresses of an address space. The address range of the address space exceeds a maximum number of addresses expected to be applied during a lifespan of the non-volatile solid-state storage. The method includes writing each of the plurality of portions of user data to the non-volatile solid-state storage such that each of the plurality of portions of user data is identified and locatable for reading via the one of the plurality of sequential, nonrepeating addresses of the address space.
Abstract:
A method for managing non-volatile memory is provided. The method includes determining at least one property of a data and determining to which type of a plurality of types of flash memory to write the data, based on the at least one property of the data. The plurality of types of flash memory includes at least two types having differing numbers of bits per cell. The method includes writing the data to a flash memory of the determined type. A flash manager and a flash storage device are provided.
Abstract:
A method of applying scheduling policies is provided. The method includes distributing user data throughout a plurality of storage nodes through erasure coding, wherein the plurality of storage nodes are housed within a single chassis coupling the storage nodes as a cluster. The method includes receiving operations relating to a non-volatile memory of one of the plurality of storage nodes into a plurality of operation queues. The method includes evaluating each of the operations in the plurality of operation queues as to benefit to the non-volatile solid-state storage according to a plurality of policies. For each channel of a plurality of channels coupling the operation queues to the non-volatile memory, the method includes iterating a selection and an execution of a next operation from the plurality of operation queues, with each next operation having a greater benefit than at least a subset of operations remaining in the operation queues.
Abstract:
A method for updates in a storage system is provided. The method includes writing identifiers, associated with data to be stored, to storage units of the storage system and writing trim records indicative of identifiers that are allowed to not exist in the storage system to the storage units. The method includes determining whether stored data corresponding to records of identifiers is valid based on the records of the identifiers and the trim records.