-
公开(公告)号:US20170075753A1
公开(公告)日:2017-03-16
申请号:US15342717
申请日:2016-11-03
Applicant: Google Inc.
Inventor: Robert Cypher , Sean Quinlan , Steven Robert Schirripa , Lidor Carmi , Christian Eric Schrock
CPC classification number: G06F11/2094 , G06F3/0619 , G06F3/0643 , G06F3/067 , G06F11/0727 , G06F11/0793 , G06F11/1076 , G06F2201/805 , G06F2211/1028
Abstract: A method of distributing data in a distributed storage system includes receiving a file and dividing the received file into chunks. The chunks are data-chunks and non-data chunks. The method further includes grouping chunks into a group and determining a distribution of the chunks of the group among storage devices of the distributed storage system based on a maintenance hierarchy of the distributed storage system. The maintenance hierarchy includes hierarchical maintenance levels and maintenance domains. Each maintenance domain has an active state or an inactive state; and each storage device is associated with at least one maintenance domain. The method also includes distributing the chunks of the group to the storage devices based on the determined distribution. The chunks of the group are distributed across multiple maintenance domains to maintain an ability to reconstruct chunks of the group when a maintenance domain is in the inactive state.
Abstract translation: 在分布式存储系统中分发数据的方法包括接收文件并将所接收的文件划分成块。 这些块是数据块和非数据块。 该方法还包括基于分布式存储系统的维护层次,将组块分组为组,并且确定分布式存储系统的存储设备中的组块的分布。 维护层次结构包括分层维护级别和维护域。 每个维护域具有活动状态或非活动状态; 并且每个存储设备与至少一个维护域相关联。 该方法还包括基于所确定的分布将组的块分配到存储设备。 组中的组块分布在多个维护域中,以便在维护域处于非活动状态时维持重组组的块的能力。
-
公开(公告)号:US09514015B2
公开(公告)日:2016-12-06
申请号:US15079095
申请日:2016-03-24
Applicant: Google Inc.
Inventor: Robert Cypher , Sean Quinlan , Steven Robert Schirripa , Lidor Carmi , Christian Eric Schrock
CPC classification number: G06F11/2094 , G06F3/0619 , G06F3/0643 , G06F3/067 , G06F11/0727 , G06F11/0793 , G06F11/1076 , G06F2201/805 , G06F2211/1028
Abstract: A method of distributing data in a distributed storage system includes receiving a file into non-transitory memory and dividing the received file into chunks. The chunks are data-chunks and non-data chunks. The method also includes grouping one or more of the data chunks and one or more of the non-data chunks in a group. One or more chunks of the group is capable of being reconstructed from other chunks of the group. The method also includes distributing the chunks of the group to storage devices of the distributed storage system based on a hierarchy of the distributed storage system. The hierarchy includes maintenance domains having active and inactive states, each storage device associated with a maintenance domain, the chunks of a group are distributed across multiple maintenance domains to maintain the ability to reconstruct chunks of the group when a maintenance domain is in an inactive state.
Abstract translation: 在分布式存储系统中分发数据的方法包括将文件接收到非暂时存储器中并将接收到的文件分割成块。 这些块是数据块和非数据块。 该方法还包括将一个或多个数据块和一组中的一个或多个非数据块分组。 该组中的一个或多个组块能够从该组的其他组块重构。 该方法还包括基于分布式存储系统的层次,将该组块分配到分布式存储系统的存储设备。 层次结构包括具有活动状态和非活动状态的维护域,每个存储设备与维护域相关联,组的块被分布在多个维护域上,以便在维护域处于非活动状态时维持重组组的块的能力 。
-
公开(公告)号:US20160203066A1
公开(公告)日:2016-07-14
申请号:US15079095
申请日:2016-03-24
Applicant: Google Inc.
Inventor: Robert Cypher , Sean Quinlan , Steven Robert Schirripa , Lidor Carmi , Christian Eric Schrock
IPC: G06F11/20
CPC classification number: G06F11/2094 , G06F3/0619 , G06F3/0643 , G06F3/067 , G06F11/0727 , G06F11/0793 , G06F11/1076 , G06F2201/805 , G06F2211/1028
Abstract: A method of distributing data in a distributed storage system includes receiving a file into non-transitory memory and dividing the received file into chunks. The chunks are data-chunks and non-data chunks. The method also includes grouping one or more of the data chunks and one or more of the non-data chunks in a group. One or more chunks of the group is capable of being reconstructed from other chunks of the group. The method also includes distributing the chunks of the group to storage devices of the distributed storage system based on a hierarchy of the distributed storage system. The hierarchy includes maintenance domains having active and inactive states, each storage device associated with a maintenance domain, the chunks of a group are distributed across multiple maintenance domains to maintain the ability to reconstruct chunks of the group when a maintenance domain is in an inactive state.
-
公开(公告)号:US09323615B2
公开(公告)日:2016-04-26
申请号:US14169322
申请日:2014-01-31
Applicant: Google Inc.
Inventor: Robert Cypher , Sean Quinlan , Steven Robert Schirripa , Lidor Carmi , Christian Eric Schrock
CPC classification number: G06F11/2094 , G06F3/0619 , G06F3/0643 , G06F3/067 , G06F11/0727 , G06F11/0793 , G06F11/1076 , G06F2201/805 , G06F2211/1028
Abstract: A method of distributing data in a distributed storage system includes receiving a file into non-transitory memory and dividing the received file into chunks. The chunks are data-chunks and non-data chunks. The method also includes grouping one or more of the data chunks and one or more of the non-data chunks in a group. One or more chunks of the group is capable of being reconstructed from other chunks of the group. The method also includes distributing the chunks of the group to storage devices of the distributed storage system based on a hierarchy of the distributed storage system. The hierarchy includes maintenance domains having active and inactive states, each storage device associated with a maintenance domain, the chunks of a group are distributed across multiple maintenance domains to maintain the ability to reconstruct chunks of the group when a maintenance domain is in an inactive state.
-
公开(公告)号:US11080140B1
公开(公告)日:2021-08-03
申请号:US15720784
申请日:2017-09-29
Applicant: Google Inc.
Inventor: Lidor Carmi , Christian Eric Schrock , Steven Robert Schirripa
IPC: G06F11/14
Abstract: A method of operating a distributed storage system, the method includes identifying unhealthy chunks of a file. The file is divided into stripes that include data chunks and non-data chunks. The method also includes identifying healthy chunks available for reconstructing the unhealthy chunks and reconstructing unhealthy data chunks before reconstructing unhealthy non-data chunks using the available healthy chunks. When the unhealthy chunk is an unhealthy word-check chunk: reconstructing the unhealthy word-check chunk using healthy word-check chunks and healthy code-check-word-check chunks; determining whether reconstruction of the unhealthy word-check chunk is possible using only healthy word-check chunks and healthy code-check-word-check chunks; and when reconstruction of the unhealthy word-check chunk is not possible using only healthy word-check chunks and healthy code-check-word-check chunks, reconstructing the unhealthy word-check chunk using any healthy chunks.
-
公开(公告)号:US09804925B1
公开(公告)日:2017-10-31
申请号:US14188773
申请日:2014-02-25
Applicant: Google Inc.
Inventor: Lidor Carmi , Christian Eric Schrock , Steven Robert Schirripa
CPC classification number: G06F11/1412
Abstract: A method of operating a distributed storage system, the method includes identifying missing chunks of a file. The file is divided into stripes that include data chunks and non-data chunks. The method also includes identifying non-missing chunks available for reconstructing the missing chunks and reconstructing missing data chunks before reconstructing missing non-data chunks using the available non-missing chunks.
-
公开(公告)号:US09747155B2
公开(公告)日:2017-08-29
申请号:US15342717
申请日:2016-11-03
Applicant: Google Inc.
Inventor: Robert Cypher , Sean Quinlan , Steven Robert Schirripa , Lidor Carmi , Christian Eric Schrock
CPC classification number: G06F11/2094 , G06F3/0619 , G06F3/0643 , G06F3/067 , G06F11/0727 , G06F11/0793 , G06F11/1076 , G06F2201/805 , G06F2211/1028
Abstract: A method of distributing data in a distributed storage system includes receiving a file and dividing the received file into chunks. The chunks are data-chunks and non-data chunks. The method further includes grouping chunks into a group and determining a distribution of the chunks of the group among storage devices of the distributed storage system based on a maintenance hierarchy of the distributed storage system. The maintenance hierarchy includes hierarchical maintenance levels and maintenance domains. Each maintenance domain has an active state or an inactive state; and each storage device is associated with at least one maintenance domain. The method also includes distributing the chunks of the group to the storage devices based on the determined distribution. The chunks of the group are distributed across multiple maintenance domains to maintain an ability to reconstruct chunks of the group when a maintenance domain is in the inactive state.
-
公开(公告)号:US20150220429A1
公开(公告)日:2015-08-06
申请号:US14169322
申请日:2014-01-31
Applicant: Google Inc.
Inventor: Robert Cypher , Sean Quinlan , Steven Robert Schirripa , Lidor Carmi , Christian Eric Schrock
IPC: G06F12/02
CPC classification number: G06F11/2094 , G06F3/0619 , G06F3/0643 , G06F3/067 , G06F11/0727 , G06F11/0793 , G06F11/1076 , G06F2201/805 , G06F2211/1028
Abstract: A method of distributing data in a distributed storage system includes receiving a file into non-transitory memory and dividing the received file into chunks. The chunks are data-chunks and non-data chunks. The method also includes grouping one or more of the data chunks and one or more of the non-data chunks in a group. One or more chunks of the group is capable of being reconstructed from other chunks of the group. The method also includes distributing the chunks of the group to storage devices of the distributed storage system based on a hierarchy of the distributed storage system. The hierarchy includes maintenance domains having active and inactive states, each storage device associated with a maintenance domain, the chunks of a group are distributed across multiple maintenance domains to maintain the ability to reconstruct chunks of the group when a maintenance domain is in an inactive state.
Abstract translation: 在分布式存储系统中分发数据的方法包括将文件接收到非暂时存储器中并将接收到的文件分割成块。 这些块是数据块和非数据块。 该方法还包括将一个或多个数据块和一组中的一个或多个非数据块分组。 该组中的一个或多个组块能够从该组的其他组块重构。 该方法还包括基于分布式存储系统的层次,将该组块分配到分布式存储系统的存储设备。 层次结构包括具有活动状态和非活动状态的维护域,每个存储设备与维护域相关联,组的块被分布在多个维护域上,以便在维护域处于非活动状态时维持重组组的块的能力 。
-
-
-
-
-
-
-