Abstract:
A method begins with a processing module receiving, from a requesting device, a data storage request that includes data for storage. The method continues with the processing module determining whether substantially identical data is currently stored in a dispersed storage network (DSN) memory. The method continues with the processing module encoding at least a portion of the data using an error coding dispersal storage function to produce a set of encoded data slices, sending the set of encoded data slices to the DSN memory for storage therein, and generating a unique retrieval matrix for the requesting device, wherein the unique retrieval matrix identifies a sub-set of encoded data slices of the set of encoded data slices for subsequent retrieval of the at least a portion of the data, when the substantially identical data is not stored in the DSN memory.
Abstract:
A method begins by a dispersed storage (DS) processing module selecting a set of distributed storage and task (DST) execution units for executing a task and determining dispersed storage error coding parameters for data. The method continues with the DS processing module dispersed storage error encoding the data in accordance with the parameters to produce a plurality of encoded data blocks and grouping the plurality of encoded data blocks into a plurality of encoded data block groupings. The method continues with the DS processing module partitioning the task into a set of partial tasks, outputting at least some of the plurality of encoded data block groupings to the set of DST execution units, and outputting the set of partial tasks to the set of DST execution units for execution of the set of partial tasks on the at least some of plurality of encoded data block groupings.
Abstract:
A method begins by a requesting entity issuing a rebuild request regarding an encoded data slice to at least some of a set of distributed storage (DS) units. In response to the rebuild request, the method continues with each of at least some of the DS units of the set of DS units generating a partial slice corresponding to the encoded data slice to be rebuilt based on one of a set of encoded data slices stored by the respective DS unit to produce an array of partial slices. The method continues with the at least some of the DS units encrypting the array of partial slices using a set of encryption keys to produce an array of encrypted partial slices. The method continues with the requesting entity rebuilding the encoded data slice from the array of encrypted partial slices.
Abstract:
A distributed storage integrity system in a dispersed storage network includes a scanning agent and a control unit. The scanning agent identifies an encoded data slice that requires rebuilding, wherein the encoded data slice is one of a plurality of encoded data slices generated from a data segment using an error encoding dispersal function. The control unit retrieves at least a number T of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function. The control unit is operable to reconstruct the data segment from at least the number T of the encoded data slices and generate a rebuilt encoded data slice from the reconstructed data segment. The scanning agent is located in a storage unit and the control unit is located in the storage unit or in a storage integrity processing unit, a dispersed storage processing unit or a dispersed storage managing unit.
Abstract:
A method begins by each of a group of write requesting modules of a dispersed storage network (DSN) generating one or more sets of write requests regarding one of a group of portions of related data, sending a group of the one or more sets of write requests to DSN memory, and sending binding information to a binding module. The method continues with the binding module processing remaining phases of the group of the one or more sets of write requests for writing the related data into the DSN memory as a single set of write requests and notifying the write requesting modules of status of the writing the related data into the DSN memory at completion of the processing of the remaining phases such that the related data is made accessible as a single piece of data when the processing of the remaining phases is successful.
Abstract:
A method begins by a processing module identifying a memory device having an expired useable memory life with respect to a legacy storage protocol. The method continues with the processing module extracting data from the memory device. The method continues with the processing module reprovisioning the memory device from the legacy storage protocol to a dispersed storage error coding storage protocol.
Abstract:
A method to generate a protocol header of a dispersed storage network (DSN) frame begins by a processing module generating a protocol class field and generating a protocol class version field. The method continues with the processing module generating an operation code (opcode) field and generating a request/response field. The method continues with the processing module generating a request number field and generating a payload length field. The method continues with the processing module outputting, in order, the protocol class field, the protocol class version field, the opcode field, the request/response field, the request number field, and the payload length field.
Abstract:
A method begins by a processing module receiving an encoded data slice to store and determining a slice length of the encoded data slice. The method continues with the processing module comparing the slice length to a plurality of bin widths, wherein each of the plurality of bin widths represents a fixed storage width of a plurality of memory bins within each of a plurality of memory containers, wherein a storage unit includes the plurality of memory containers. The method continues with the processing module selecting one of the plurality of memory containers based on the comparing to produce a selected memory container, identifying an available bin of the plurality of bins of the selected memory container, and storing the encoded data slice in the available bin.
Abstract:
A method begins by a dispersed storage (DS) processing module of a dispersed storage network (DSN), when in a rebuilding mode, determining a level of rebuilding urgency, where the level of rebuilding urgency is based on an inversely proportional function of a level of error encoded redundancy for error encoded data giving rise to the rebuilding mode. When the level of rebuilding urgency is at or above a critical level, the method continues with the DS processing module placing a set of storage units of the DSN in a critical rebuilding mode, where, when in the critical rebuilding mode, the set of storage units prioritize rebuilding of the error encoded data giving rise to the rebuilding mode over other error encoded data operations.
Abstract:
A method begins by first and second client devices transmitting first and second sets of write requests to storage units. The method continues with a storage unit sending a write response message to the first and second client devices, where the write response message indicates a storage unit score value for one of the client devices. The method continues with the first and second client devices interpreting the storage unit score values from the storage units to determine which client device has write priority. The method continues with the client device that has the write priority sending a set of next-phase write requests to the storage units. The method continues with the other client device that does not have the write priority sending a set of rollback requests to the storage units.