REMOVABLE MEDIA BASED OBJECT STORE

    公开(公告)号:US20210294514A1

    公开(公告)日:2021-09-23

    申请号:US17341229

    申请日:2021-06-07

    IPC分类号: G06F3/06

    摘要: Embodiments disclosed herein provide systems, methods, and computer-readable media to implement an object store with removable storage media. In a particular embodiment, a method provides identifying first data for storage on a first removable storage medium and designating at least a portion of the first data to a first data object. The method further provides determining a first location where to store the first data object in a first value store partition of the first removable storage medium and writing the first data object to the first location. Also, the method provides writing a first key that identifies the first data object and indicates the first location to a first key store partition of the first removable storage medium.

    Joint de-duplication-erasure coded distributed storage

    公开(公告)号:US10853187B2

    公开(公告)日:2020-12-01

    申请号:US16285320

    申请日:2019-02-26

    摘要: Methods and apparatus deduplicate and erasure code a message in a data storage system. One example apparatus includes a first chunking circuit that generates a set of data chunks from a message, an outer precoding circuit that generates a set of precoded data chunks and a set of parity symbols from the set of data chunks, a second chunking circuit that generates a set of chunked parity symbols from the set of parity symbols, a deduplication circuit that generates a set of deduplicated data chunks by deduplicating the set of precoded chunks or the set of chunked parity symbols, an unequal error protection (UEP) circuit that generates an encoded message from the set of deduplicated data chunks, and a storage circuit that controls the data storage system to store the set of deduplicated data chunks, the set of parity symbols, or the encoded message.

    FILTER FILE SYSTEM WITH INODE NUMBER AS PRIMARY DATABASE KEY

    公开(公告)号:US20200065293A1

    公开(公告)日:2020-02-27

    申请号:US16669574

    申请日:2019-10-31

    IPC分类号: G06F16/13 G06F16/17 G06F16/16

    摘要: Example apparatus and methods identify files that are so small or so large that they compromise the efficient operation of a file system that uses re-assignable one-to-one inodes and inode numbers. Small files are aggregated into collections of files and large files are subdivided into collections of smaller files. Information for locating multiple related files with fewer lookups is generated and stored in a folder. An inode having a new type of inode number is then created. The new type of inode number encodes information for finding the folder. The encoded information may include a folder identifier that acts as a primary key into a database that is configured to locate a member of the aggregated or subdivided files with a single lookup. A filter file system may be updated with the new inode. The new inode number is unique within the filter file system and may not be re-assigned.

    Data Deduplication With Adaptive Erasure Code Redundancy
    5.
    发明申请
    Data Deduplication With Adaptive Erasure Code Redundancy 有权
    具有自适应擦除代码冗余的重复数据删除

    公开(公告)号:US20160013815A1

    公开(公告)日:2016-01-14

    申请号:US14326774

    申请日:2014-07-09

    IPC分类号: H03M13/37 G06F11/14

    摘要: Example apparatus and methods combine erasure coding with data deduplication to simultaneously reduce the overall redundancy in data while increasing the redundancy of unique data. In one embodiment, an efficient representation of a data set is produced by deduplication. The efficient representation reduces duplicate data in the data set. Redundancy is then added back into the data set using erasure coding. The redundancy that is added back in adds protection to the unique data associated with the efficient representation. How much redundancy is added back in and what type of redundancy is added back in may be controlled based on an attribute (e.g., value, reference count, symbol size, number of symbols) of the unique data. Decisions concerning how much and what type of redundancy to add back in may be adapted over time based, for example, on observations of the efficiency of the overall system.

    摘要翻译: 将擦除编码与重复数据消除相结合的示例性设备和方法可同时减少数据的总体冗余,同时增加唯一数据的冗余度。 在一个实施例中,通过重复数据消除产生数据集的有效表示。 有效的表示可以减少数据集中的重复数据。 然后使用擦除编码将冗余加回到数据集中。 添加的冗余增加了与有效表示相关联的唯一数据的保护。 可以基于唯一数据的属性(例如,值,引用计数,符号大小,符号数量)来控制多少冗余被添加回来以及添加哪种类型的冗余。 例如,关于整个系统的效率的观察,可以根据时间来适应关于多少和什么类型的冗余的决定。

    Removable media based object store

    公开(公告)号:US11042299B2

    公开(公告)日:2021-06-22

    申请号:US15340691

    申请日:2016-11-01

    摘要: Embodiments disclosed herein provide systems, methods, and computer-readable media to implement an object store with removable storage media. In a particular embodiment, a method provides identifying first data for storage on a first removable storage medium and designating at least a portion of the first data to a first data object. The method further provides determining a first location where to store the first data object in a first value store partition of the first removable storage medium and writing the first data object to the first location. Also, the method provides writing a first key that identifies the first data object and indicates the first location to a first key store partition of the first removable storage medium.

    Garbage collection and defragmentation for solid state drives (SSD) and shingled magnetic recording (SMR) drives

    公开(公告)号:US10474571B2

    公开(公告)日:2019-11-12

    申请号:US15618723

    申请日:2017-06-09

    IPC分类号: H03M13/00 G06F12/02 H03M13/37

    摘要: Example apparatus and methods provide improved reclamation, garbage collection (GC) and defragmentation (defrag) for data storage devices including solid state drives (SSD) or shingled magnetic recording (SMR) drives. An erasure code (EC) layer that facilitates logically or physically erasing data from the SSD or SMR as a comprehensive GC or defrag is added to the SSD or SMR. Erased data may be selectively recreated from the EC layer as needed. Pre-planned EC write zones may be established to further optimize GC and defrag. Recreated data may be written to selected locations to further optimize SSD and SMR performance. Erasure code data may be distributed to co-operating devices to further improve GC or defrag. Example apparatus and methods may also facilitate writing data to an SMR drive using tape or VTL applications or processes and providing a pseudo virtual tape library on the SMR drive.

    Joint de-duplication-erasure coded distributed storage

    公开(公告)号:US10318389B2

    公开(公告)日:2019-06-11

    申请号:US15211208

    申请日:2016-07-15

    摘要: Methods and apparatus deduplicate and erasure code a message in a data storage system. One example apparatus includes a first chunking circuit that generates a set of data chunks from a message, an outer precoding circuit that generates a set of precoded data chunks and a set of parity symbols from the set of data chunks, a second chunking circuit that generates a set of chunked parity symbols from the set of parity symbols, a deduplication circuit that generates a set of deduplicated data chunks by deduplicating the set of precoded chunks or the set of chunked parity symbols, an unequal error protection (UEP) circuit that generates an encoded message from the set of deduplicated data chunks, and a storage circuit that controls the data storage system to store the set of deduplicated data chunks, the set of parity symbols, or the encoded message.

    Messaging policy controlled email de-duplication

    公开(公告)号:US09876746B2

    公开(公告)日:2018-01-23

    申请号:US14515705

    申请日:2014-10-16

    IPC分类号: G06F15/16 H04L12/58 G06F17/30

    摘要: Example methods and apparatus associated with a messaging policy controlled email deduplication are provided. In one example a messaging policy is accessed. It is determined whether a received message complies with the policy based on rules of the messaging policy. If a message complies with the messaging policy, the message is displayed. If the message does not comply with the messaging policy, it is determined whether the message is duplicative. If the message is deemed duplicative it is not displayed. Conversely, if the message is not deemed duplicative it is displayed.