摘要:
This invention provides a defragmentation utility that works on-line in parallel with other file system activities. Thus, it avoids making the file system unavailable for periods of time which would, if not for this invention, slow down data communication exchange and the execution of other tasks dependent upon the data. In particular, this invention, steps through all of the valid inodes finding each of the fragments. The defragmentation engine decides which fragments must remain in their current location and which fragments should migrate to another disk block sub-block location. Since the data blocks span across multiple disks, for each valid disk of the file system a set of disk blocks are constructed that are chosen to be filled, herein called plates. When the plates become full or reach a certain fullness, they are removed from the set and replaced by other disk blocks. When a disk block is removed from the plate set, it is moved to a “done” list as it is considered “full”. While a disk block is in the done list the fragments that belong to that block are not allowed to migrate. Thus, a defragmentation utility that works on-line, avoids locking data structures for long periods of time, is memory efficient, uses sub-blocks for fragment analysis and migration as well as minimizes data movements has been summarized. This utility thereby provides a transparent defragmentation function that operates in the background seamlessly with other system file operations.
摘要:
File system backups are carried out by first generating a list of inodes including associated inode numbers in inode number order that have changed since the last backup operation. A table which has file names and inode numbers for all of the files currently in the file system is also generated. This list is sorted by inode number and the table and list are merged to provide a structure for determining which files are to be backed up. This means that relevant inodes and file names are now provided in a single entity. It is also noted that the structure that results from the merge operation is particularly suitable for being read in blocks which thus permits the backup operation to be carried out in parallel. The task of backing up files is also preferably partitioned by file size or other criteria as opposed to being partitioned simply by the number of files assigned to be backed up by any one processor in a distributed or parallel data processing system.
摘要:
A data validation method comprises associating a validation chunk with one or more data chunks stored on one or more storage mediums. The validation chunk comprises validation information for verifying whether data recorded in a subdivision of the one or more data chunks is lost. A request is received for an operation to be performed on first data stored in a first subdivision of a first data chunk from among said one or more data chunks. One or more portions of the validation information are examined to determine whether the data recorded in said first subdivision is deemed to be lost.
摘要:
Data validation systems and methods are provided. Data is recorded in N data chunks on one or more storage mediums. A first validation chunk independently associated with said N data chunks comprises first validation information for verifying accuracy of data recorded in said N data chunks. The first validation chunk is associated with a first validation appendix comprising second validation information, wherein the first validation appendix is stored on a first storage medium independent of said one or more storage mediums.
摘要:
Concurrent access to data is managed through concurrency control techniques. Various types of techniques are employed to manage the access, including locking-based techniques and non-locking-based techniques. A dynamic switch from one type of concurrency control technique (e.g., a locking-based technique) to a different type of concurrency control technique (e.g., a non-locking-based technique) is enabled. This switching is based on access patterns and/or application requirements for each file. The switching allows enhanced performance for both coarse-grain sharing and fine-grain sharing of data.
摘要:
A data validation method comprises associating a validation chunk with one or more data chunks stored on one or more storage mediums. The validation chunk comprises validation information for verifying whether data recorded in a subdivision of the one or more data chunks is lost. A request is received for an operation to be performed on first data stored in a first subdivision of a first data chunk from among said one or more data chunks. One or more portions of the validation information are examined to determine whether the data recorded in said first subdivision is deemed to be lost.
摘要:
The decision to prefetch inodes is based upon the detecting of access patterns that would benefit from such a prefetch. Once the decision to prefetch is made, a plurality of inodes are prefetched in parallel. Further, the prefetching of inodes is paced, such that the prefetching substantially matches the speed at which an application requests inodes.
摘要:
Data validation systems and methods are provided. Data is recorded in N data chunks on one or more storage mediums. A first validation chunk independently associated with said N data chunks comprises first validation information for verifying accuracy of data recorded in said N data chunks. The first validation chunk is associated with a first validation appendix comprising second validation information, wherein the first validation appendix is stored on a first storage medium independent of said one or more storage mediums.
摘要:
A system, method and computer readable medium for providing a file system snapshot with ditto address feature is disclosed. The method includes generating a file system snapshot wherein the snapshot is substantially empty. When metadata associated with a source file included in the snapshot is modified, the inode corresponding to the source file is copied to a shadow inode in the snapshot. When information is appended to a data block associated with a source file included in the snapshot, the inode corresponding to the source file is also copied to a shadow inode in the snapshot. When information in a data block associated with a source file included in the snapshot is overwritten or deleted, the inode corresponding to the source file is copied to a shadow inode in the snapshot and the original data block corresponding to the source file is copied to the snapshot.
摘要:
Space is allocated on data storage devices in proportion to weights associated with the storage devices. The weights can be dynamically adjusted at any time in order to accommodate changes in the system and to better utilize the storage devices. The technique used to perform the allocating is independent of the weights used by the allocating. Further, the allocation technique can accommodate general purpose data streams having varying lengths and/or varying access patterns, as well as special purpose data streams, such as video streams.