GROUPING OF OBJECTS IN A DISTRIBUTED STORAGE SYSTEM BASED ON JOURNALS AND PLACEMENT POLICIES
    1.
    发明公开
    GROUPING OF OBJECTS IN A DISTRIBUTED STORAGE SYSTEM BASED ON JOURNALS AND PLACEMENT POLICIES 审中-公开
    基于波THONG与布局规则的分布式存储系统对象分组

    公开(公告)号:EP3014487A1

    公开(公告)日:2016-05-04

    申请号:EP14742036.8

    申请日:2014-06-23

    申请人: Google Inc.

    IPC分类号: G06F17/30 G06F3/06 H04L29/08

    摘要: Managing placement of object replicas is performed at a first instance of a distributed storage system. One or more journals are opened for storage of object chunks. Each journal is associated with a single placement policy. A first object is received comprising at least a first object chunk. The first object is associated with a first placement policy. The first object chunk is stored in a first journal whose associated placement policy matches the first placement policy. The first journal stores only object chunks for objects whose placement policies match the first placement policy. For the first journal, the receiving and storing operations are repeated for multiple objects whose associated placement policies match the first placement policy, until a first termination condition occurs. Then, the first journal is closed. Subsequently, the first journal is replicated to a second instance of the distributed storage system according to the first placement policy.

    ASYNCHRONOUS DISTRIBUTED OBJECT UPLOADING FOR REPLICATED CONTENT ADDRESSABLE STORAGE CLUSTERS
    2.
    发明公开
    ASYNCHRONOUS DISTRIBUTED OBJECT UPLOADING FOR REPLICATED CONTENT ADDRESSABLE STORAGE CLUSTERS 审中-公开
    异步分布式对象UPLOAD复制的内容寻址存储集群

    公开(公告)号:EP2422282A1

    公开(公告)日:2012-02-29

    申请号:EP10716169.7

    申请日:2010-04-16

    申请人: Google Inc.

    IPC分类号: G06F17/30

    摘要: A method is performed by two or more devices of a group of devices in a distributed data replication system. The method includes receiving, at the two or more devices, a group of chunks having a same unique temporary identifier, where the group of chunks comprises an object to be uploaded; creating an entry for the object in a replicated index, where the entry is keyed by the unique temporary identifier, and where the replicated index is replicated at each of the two or more devices; and determining, by an initiating device of the two or more devices, that a union of the group of chunks contains all data of the object. The method also includes calculating a content-based identifier to the object; creating another entry for the object in the replicated index, where the other entry is keyed by the content-based identifier; and updating the replicated index to point from the unique temporary identifier to the content-based identifier.

    ASYNCHRONOUS DISTRIBUTED DE-DUPLICATION FOR REPLICATED CONTENT ADDRESSABLE STORAGE CLUSTERS
    5.
    发明公开
    ASYNCHRONOUS DISTRIBUTED DE-DUPLICATION FOR REPLICATED CONTENT ADDRESSABLE STORAGE CLUSTERS 审中-公开
    ASYNCHRONE VERTEILTE ENTUPLIKATIONFÜRREPLIZIERTE INHALTSADRESSIERBARE SPEICHERCLUSTER

    公开(公告)号:EP2368199A1

    公开(公告)日:2011-09-28

    申请号:EP09799831.4

    申请日:2009-12-22

    申请人: Google Inc.

    IPC分类号: G06F17/30

    摘要: A method is performed by a device of a group of devices in a distributed data replication system. The method includes storing an index of objects in the distributed data replication system, the index being replicated while the objects are stored locally by the plurality of devices in the distributed data replication system. The method also includes conducting a scan of at least a portion of the index and identifying a redundant replica(s) of the at least one of the objects based on the scan of the index. The method further includes de-duplicating the redundant replica(s), and updating the index to reflect the status of the redundant replica.

    摘要翻译: 一种方法由分布式数据复制系统中的一组设备的设备执行。 该方法包括将对象索引存储在分布式数据复制系统中,索引被复制,同时对象由分布式数据复制系统中的多个设备本地存储。 该方法还包括对索引的至少一部分进行扫描,并且基于索引的扫描来识别所述至少一个对象的冗余副本。 该方法还包括解除复制冗余副本,以及更新索引以反映冗余副本的状态。

    HIERARCHICAL CHUNKING OF OBJECTS IN A DISTRIBUTED STORAGE SYSTEM
    8.
    发明公开
    HIERARCHICAL CHUNKING OF OBJECTS IN A DISTRIBUTED STORAGE SYSTEM 审中-公开
    物体在分布式存储系统分层BREAKDOWN

    公开(公告)号:EP3087513A1

    公开(公告)日:2016-11-02

    申请号:EP14874957.5

    申请日:2014-12-24

    申请人: Google Inc.

    IPC分类号: G06F17/30

    摘要: Placement of object replicas in a distributed storage system includes, at a first instance, opening a journal for storage of object chunks. Each journal is associated with a single placement policy. An object is received, which comprises a chunk. The object has a placement policy, and the chunk comprises a plurality of storage blocks. The blocks are stored in a journal that matches the placement policy. Global metadata for the object is stored, which includes a list of chunks for the object. Local metadata for the chunk is stored, which includes a block list identifying each block of the plurality of blocks. The local metadata is associated with the journal. The journal is later closed. The journal is subsequently replicated to a second instance according to the placement policy. The global metadata is updated to reflect the replication, whereas the local metadata is unchanged by the replication.