公开(公告)号：US20160203066A1

公开(公告)日：2016-07-14

申请号：US15079095

申请日：2016-03-24

Applicant: Google Inc.

Inventor： Robert Cypher ,  Sean Quinlan ,  Steven Robert Schirripa ,  Lidor Carmi ,  Christian Eric Schrock

IPC: G06F11/20

CPC classification number: G06F11/2094 ,  G06F3/0619 ,  G06F3/0643 ,  G06F3/067 ,  G06F11/0727 ,  G06F11/0793 ,  G06F11/1076 ,  G06F2201/805 ,  G06F2211/1028

Abstract: A method of distributing data in a distributed storage system includes receiving a file into non-transitory memory and dividing the received file into chunks. The chunks are data-chunks and non-data chunks. The method also includes grouping one or more of the data chunks and one or more of the non-data chunks in a group. One or more chunks of the group is capable of being reconstructed from other chunks of the group. The method also includes distributing the chunks of the group to storage devices of the distributed storage system based on a hierarchy of the distributed storage system. The hierarchy includes maintenance domains having active and inactive states, each storage device associated with a maintenance domain, the chunks of a group are distributed across multiple maintenance domains to maintain the ability to reconstruct chunks of the group when a maintenance domain is in an inactive state.

公开(公告)号：US20160179621A1

公开(公告)日：2016-06-23

申请号：US15054780

申请日：2016-02-26

Applicant: Google Inc.

Inventor： Steven Robert Schirripa ,  Christian Eric Schrock ,  Robert Cypher ,  Sean Quinlan

IPC: G06F11/10 ,  G06F3/06

CPC classification number: G06F11/1464 ,  G06F3/0619 ,  G06F3/064 ,  G06F3/0644 ,  G06F3/067 ,  G06F11/00 ,  G06F11/008 ,  G06F11/1004 ,  G06F11/1088 ,  G06F11/1092 ,  G06F11/1461 ,  G06F11/1469 ,  G06F11/2094 ,  G06F17/30215 ,  H03M13/1515

Abstract: A method of prioritizing data for recovery in a distributed storage system includes, for each stripe of a file having chunks, determining whether the stripe comprises high-availability chunks or low-availability chunks and determining an effective redundancy value for each stripe. The effective redundancy value is based on the chunks and any system domains associated with the corresponding stripe. The distributed storage system has a system hierarchy including system domains. Chunks of a stripe associated with a system domain in an active state are accessible, whereas chunks of a stripe associated with a system domain in an inactive state are inaccessible. The method also includes reconstructing substantially immediately inaccessible, high-availability chunks having an effective redundancy value less than a threshold effective redundancy value and reconstructing the inaccessible low-availability and other inaccessible high-availability chunks, after a threshold period of time.

公开(公告)号：US09323615B2

公开(公告)日：2016-04-26

申请号：US14169322

申请日：2014-01-31

Applicant: Google Inc.

Inventor： Robert Cypher ,  Sean Quinlan ,  Steven Robert Schirripa ,  Lidor Carmi ,  Christian Eric Schrock

IPC: G06F12/02 ,  G06F11/10

CPC classification number: G06F11/2094 ,  G06F3/0619 ,  G06F3/0643 ,  G06F3/067 ,  G06F11/0727 ,  G06F11/0793 ,  G06F11/1076 ,  G06F2201/805 ,  G06F2211/1028

Abstract: A method of distributing data in a distributed storage system includes receiving a file into non-transitory memory and dividing the received file into chunks. The chunks are data-chunks and non-data chunks. The method also includes grouping one or more of the data chunks and one or more of the non-data chunks in a group. One or more chunks of the group is capable of being reconstructed from other chunks of the group. The method also includes distributing the chunks of the group to storage devices of the distributed storage system based on a hierarchy of the distributed storage system. The hierarchy includes maintenance domains having active and inactive states, each storage device associated with a maintenance domain, the chunks of a group are distributed across multiple maintenance domains to maintain the ability to reconstruct chunks of the group when a maintenance domain is in an inactive state.

公开(公告)号：US09213576B2

公开(公告)日：2015-12-15

申请号：US14169357

申请日：2014-01-31

Applicant: Google Inc.

Inventor： Robert Cypher ,  Peter Dahl ,  Steven Robert Schirripa

IPC: G06F9/46 ,  G06F9/50 ,  G06F9/48

CPC classification number: G06F9/5011 ,  G06F9/4881 ,  G06F9/50 ,  G06F9/5061 ,  G06F9/5088 ,  G06F2209/503

Abstract: A method includes identifying high-availability jobs and low-availability jobs that demand usage of resources of a distributed system. The method includes determining a first quota of the resources available to low-availability jobs as a quantity of the resources available during normal operations, and determining a second quota of the resources available to high-availability jobs as a quantity of the resources available during normal operations minus a quantity of the resources lost due to a tolerated event. The method includes executing the jobs on the distributed system and constraining a total usage of the resources by both the high-availability jobs and the low-availability jobs to the quantity of the resources available during normal operations.

Abstract translation: 一种方法包括识别需要使用分布式系统资源的高可用性作业和低可用性作业。 该方法包括将可用于低可用性作业的资源的第一配额确定为在正常操作期间可用的资源的数量，以及将可用于高可用性作业的资源的第二配额确定为正常期间可用资源的数量 操作减去由于容忍事件而丢失的资源的数量。 该方法包括在分布式系统上执行作业，并将高可用性作业和低可用性作业的资源的总使用限制在正常操作期间可用资源的数量。