Prioritizing Data Reconstruction in Distributed Storage Systems
    11.
    发明申请
    Prioritizing Data Reconstruction in Distributed Storage Systems 有权
    分布式存储系统中的数据重建优先级

    公开(公告)号:US20150220398A1

    公开(公告)日:2015-08-06

    申请号:US14169274

    申请日:2014-01-31

    Applicant: Google Inc.

    Abstract: A method of prioritizing data for recovery in a distributed storage system includes, for each stripe of a file having chunks, determining whether the stripe comprises high-availability chunks or low-availability chunks and determining an effective redundancy value for each stripe. The effective redundancy value is based on the chunks and any system domains associated with the corresponding stripe. The distributed storage system has a system hierarchy including system domains. Chunks of a stripe associated with a system domain in an active state are accessible, whereas chunks of a stripe associated with a system domain in an inactive state are inaccessible. The method also includes reconstructing substantially immediately inaccessible, high-availability chunks having an effective redundancy value less than a threshold effective redundancy value and reconstructing the inaccessible low-availability and other inaccessible high-availability chunks, after a threshold period of time.

    Abstract translation: 对于分布式存储系统中用于恢复的数据进行优先排序的方法包括:对于具有块的文件的每个条带,确定条带是否包括高可用性块或低可用性块,并确定每个条带的有效冗余值。 有效的冗余值基于与相应条带相关联的块和任何系统域。 分布式存储系统具有系统层次结构,包括系统域。 与处于活动状态的系统域相关联的条带的块可访问,而与处于非活动状态的系统域相关联的条带的块不可访问。 该方法还包括在阈值时间段之后重建具有小于阈值有效冗余度值的有效冗余度的基本上立即不可访问的高可用性块并且重建不可访问的低可用性和其他不可访问的高可用性块。

    Data reconstruction in distributed storage systems

    公开(公告)号:US11080140B1

    公开(公告)日:2021-08-03

    申请号:US15720784

    申请日:2017-09-29

    Applicant: Google Inc.

    Abstract: A method of operating a distributed storage system, the method includes identifying unhealthy chunks of a file. The file is divided into stripes that include data chunks and non-data chunks. The method also includes identifying healthy chunks available for reconstructing the unhealthy chunks and reconstructing unhealthy data chunks before reconstructing unhealthy non-data chunks using the available healthy chunks. When the unhealthy chunk is an unhealthy word-check chunk: reconstructing the unhealthy word-check chunk using healthy word-check chunks and healthy code-check-word-check chunks; determining whether reconstruction of the unhealthy word-check chunk is possible using only healthy word-check chunks and healthy code-check-word-check chunks; and when reconstruction of the unhealthy word-check chunk is not possible using only healthy word-check chunks and healthy code-check-word-check chunks, reconstructing the unhealthy word-check chunk using any healthy chunks.

    Efficient Resource Utilization in Data Centers

    公开(公告)号:US20160077883A1

    公开(公告)日:2016-03-17

    申请号:US14948426

    申请日:2015-11-23

    Applicant: Google Inc.

    Abstract: A method includes identifying high-availability jobs and low-availability jobs that demand usage of resources of a distributed system. The method includes determining a first quota of the resources available to low-availability jobs as a quantity of the resources available during normal operations, and determining a second quota of the resources available to high-availability jobs as a quantity of the resources available during normal operations minus a quantity of the resources lost due to a tolerated event. The method includes executing the jobs on the distributed system and constraining a total usage of the resources by both the high-availability jobs and the low-availability jobs to the quantity of the resources available during normal operations.

    Efficient Data Reads From Distributed Storage Systems
    16.
    发明申请
    Efficient Data Reads From Distributed Storage Systems 有权
    分布式存储系统的高效数据读取

    公开(公告)号:US20150220429A1

    公开(公告)日:2015-08-06

    申请号:US14169322

    申请日:2014-01-31

    Applicant: Google Inc.

    Abstract: A method of distributing data in a distributed storage system includes receiving a file into non-transitory memory and dividing the received file into chunks. The chunks are data-chunks and non-data chunks. The method also includes grouping one or more of the data chunks and one or more of the non-data chunks in a group. One or more chunks of the group is capable of being reconstructed from other chunks of the group. The method also includes distributing the chunks of the group to storage devices of the distributed storage system based on a hierarchy of the distributed storage system. The hierarchy includes maintenance domains having active and inactive states, each storage device associated with a maintenance domain, the chunks of a group are distributed across multiple maintenance domains to maintain the ability to reconstruct chunks of the group when a maintenance domain is in an inactive state.

    Abstract translation: 在分布式存储系统中分发数据的方法包括将文件接收到非暂时存储器中并将接收到的文件分割成块。 这些块是数据块和非数据块。 该方法还包括将一个或多个数据块和一组中的一个或多个非数据块分组。 该组中的一个或多个组块能够从该组的其他组块重构。 该方法还包括基于分布式存储系统的层次,将该组块分配到分布式存储系统的存储设备。 层次结构包括具有活动状态和非活动状态的维护域,每个存储设备与维护域相关联,组的块被分布在多个维护域上,以便在维护域处于非活动状态时维持重组组的块的能力 。

    Preventing unnecessary data recovery

    公开(公告)号:US09898360B1

    公开(公告)日:2018-02-20

    申请号:US14980633

    申请日:2015-12-28

    Applicant: Google Inc.

    Abstract: A method that prevents unnecessary data recovery includes receiving, at a data processing device, a status of a resource of a distributed system. When the status of the resource indicates a resource failure, the method includes executing instructions on the data processing device to determine whether the resource failure is correlated to any other resource failures within the distributed system. When the resource failure is correlated to other resource failures within the distributed system, the method includes delaying execution on the data processing device of a remedial action associated with the resource. However, when the resource failure is uncorrelated to other resource failures within the distributed system, the method includes initiating execution on the data processing device of the remedial action associated with the resource.

    Efficient resource utilization in data centers

    公开(公告)号:US09823948B2

    公开(公告)日:2017-11-21

    申请号:US14948426

    申请日:2015-11-23

    Applicant: Google Inc.

    Abstract: A method includes identifying high-availability jobs and low-availability jobs that demand usage of resources of a distributed system. The method includes determining a first quota of the resources available to low-availability jobs as a quantity of the resources available during normal operations, and determining a second quota of the resources available to high-availability jobs as a quantity of the resources available during normal operations minus a quantity of the resources lost due to a tolerated event. The method includes executing the jobs on the distributed system and constraining a total usage of the resources by both the high-availability jobs and the low-availability jobs to the quantity of the resources available during normal operations.

    Prioritizing data reconstruction in distributed storage systems
    19.
    发明授权
    Prioritizing data reconstruction in distributed storage systems 有权
    分布式存储系统中数据重建的优先级

    公开(公告)号:US09535790B2

    公开(公告)日:2017-01-03

    申请号:US15054780

    申请日:2016-02-26

    Applicant: Google Inc.

    Abstract: A method of prioritizing data for recovery in a distributed storage system includes, for each stripe of a file having chunks, determining whether the stripe comprises high-availability chunks or low-availability chunks and determining an effective redundancy value for each stripe. The effective redundancy value is based on the chunks and any system domains associated with the corresponding stripe. The distributed storage system has a system hierarchy including system domains. Chunks of a stripe associated with a system domain in an active state are accessible, whereas chunks of a stripe associated with a system domain in an inactive state are inaccessible. The method also includes reconstructing substantially immediately inaccessible, high-availability chunks having an effective redundancy value less than a threshold effective redundancy value and reconstructing the inaccessible low-availability and other inaccessible high-availability chunks, after a threshold period of time.

    Abstract translation: 对于分布式存储系统中用于恢复的数据进行优先排序的方法包括:对于具有块的文件的每个条带,确定条带是否包括高可用性块或低可用性块,并确定每个条带的有效冗余值。 有效的冗余值基于与相应条带相关联的块和任何系统域。 分布式存储系统具有系统层次结构,包括系统域。 与处于活动状态的系统域相关联的条带的块可访问,而与处于非活动状态的系统域相关联的条带的块不可访问。 该方法还包括在阈值时间段之后重建具有小于阈值有效冗余度值的有效冗余度的基本上立即不可访问的高可用性块并且重建不可访问的低可用性和其他不可访问的高可用性块。

    Efficient data reads from distributed storage systems
    20.
    发明授权
    Efficient data reads from distributed storage systems 有权
    从分布式存储系统读取高效数据

    公开(公告)号:US09514015B2

    公开(公告)日:2016-12-06

    申请号:US15079095

    申请日:2016-03-24

    Applicant: Google Inc.

    Abstract: A method of distributing data in a distributed storage system includes receiving a file into non-transitory memory and dividing the received file into chunks. The chunks are data-chunks and non-data chunks. The method also includes grouping one or more of the data chunks and one or more of the non-data chunks in a group. One or more chunks of the group is capable of being reconstructed from other chunks of the group. The method also includes distributing the chunks of the group to storage devices of the distributed storage system based on a hierarchy of the distributed storage system. The hierarchy includes maintenance domains having active and inactive states, each storage device associated with a maintenance domain, the chunks of a group are distributed across multiple maintenance domains to maintain the ability to reconstruct chunks of the group when a maintenance domain is in an inactive state.

    Abstract translation: 在分布式存储系统中分发数据的方法包括将文件接收到非暂时存储器中并将接收到的文件分割成块。 这些块是数据块和非数据块。 该方法还包括将一个或多个数据块和一组中的一个或多个非数据块分组。 该组中的一个或多个组块能够从该组的其他组块重构。 该方法还包括基于分布式存储系统的层次,将该组块分配到分布式存储系统的存储设备。 层次结构包括具有活动状态和非活动状态的维护域,每个存储设备与维护域相关联,组的块被分布在多个维护域上,以便在维护域处于非活动状态时维持重组组的块的能力 。

Patent Agency Ranking