Distributing Data on Distributed Storage Systems

    公开(公告)号:US20240338279A1

    公开(公告)日:2024-10-10

    申请号:US18746351

    申请日:2024-06-18

    Applicant: Google LLC

    CPC classification number: G06F11/1435 G06F16/1748 G06F16/182 G06F16/278

    Abstract: A method of distributing data in a distributed storage system includes receiving a file, dividing the received file into chunks, and determining a distribution of the chunks among storage devices of the distributed storage system based on a maintenance hierarchy of the distributed storage system. The maintenance hierarchy includes maintenance levels, and each maintenance level includes one or more maintenance units. Each maintenance unit has an active state and an inactive state. Moreover, each storage device is associated with a maintenance unit. The determining of the distribution of the chunks includes identifying a random selection of the storage devices matching a number of chunks of the file and being capable of maintaining accessibility of the file when one or more maintenance units are in an inactive state. The method also includes distributing the chunks to storage devices of the distributed storage system according to the determined distribution.

    Systems and methods for physical capacity estimation of logical space units

    公开(公告)号:US12045203B2

    公开(公告)日:2024-07-23

    申请号:US17304106

    申请日:2021-06-14

    CPC classification number: G06F16/1748 G06F16/1727

    Abstract: Systems and methods of determining physical capacity of logical space units are disclosed. The method populates a first smart filter to track a physical capacity of a first logical space unit (LSU). The method adds fingerprints from the first LSU to register(s) of the first smart filter. The method populates a second smart filter to track fingerprints deleted by garbage collection (GC). The method adds the deleted fingerprints to register(s) of the second smart filter. Using the first and second smart filters, the method determines an intersection cardinality of the first LSU and the deleted fingerprints. The method determines a cardinality of unique fingerprints in the first LSU based on the intersection cardinality of the first LSU and the deleted fingerprints. The method determines the physical capacity of the first LSU based at least on the cardinality of unique fingerprints in the first LSU.

    Fast recovery for replication corruptions

    公开(公告)号:US11995042B1

    公开(公告)日:2024-05-28

    申请号:US18153339

    申请日:2023-01-11

    Abstract: Upon completing replicating a file set from a source to destination, a snapshot taken of the file set is maintained at the source. The file set includes a namespace file having metadata and other files having content data. Verification is started on the file set replicated to the destination. While the verification is in-progress, a next replication cycle is started. Upon detecting corruption in the namespace file, the next replication cycle is paused and any changes to the file set are rolled back. The snapshot being maintained at the source is fetched. The namespace file having the corruption is replaced with the namespace file from the snapshot being maintained at the source. Other files in the file set having the content data are not replaced.

Patent Agency Ranking