公开(公告)号：US09477679B2

公开(公告)日：2016-10-25

申请号：US14033255

申请日：2013-09-20

Applicant: GOOGLE INC.

Inventor： Murray M. Stokely ,  Arif Merchant

IPC: G06F17/30

CPC classification number: G06F17/30138 ,  G06F3/0617 ,  G06F3/0643 ,  G06F3/067 ,  G06F17/30194 ,  G06F17/30212

Abstract: Methods to determine and automatically recommend or adjust configuration parameters for storing files in large-scale distributed storage systems are disclosed. These methods may receive file metadata and trace data that allows the system to identify file access patterns. Additionally, the methods may receive information about distributed storage systems in a datacenter. This information can be used to choose storage parameters on a per-file basis for storing files.

Abstract translation: 公开了确定并自动推荐或调整用于在大规模分布式存储系统中存储文件的配置参数的方法。 这些方法可以接收允许系统识别文件访问模式的文件元数据和跟踪数据。 此外，这些方法可以接收关于数据中心中的分布式存储系统的信息。 该信息可用于根据每个文件选择存储参数以存储文件。

公开(公告)号：US20170011059A1

公开(公告)日：2017-01-12

申请号：US15271739

申请日：2016-09-21

Applicant: Google Inc.

Inventor： Murray M. Stokely ,  Arif Merchant

IPC: G06F17/30 ,  G06F3/06

CPC classification number: G06F16/1727 ,  G06F3/0617 ,  G06F3/0643 ,  G06F3/067 ,  G06F16/182 ,  G06F16/184

Abstract: A method includes receiving trace data representing access information about files stored in a large-scale distributed storage system, identifying file access patterns based on the trace data, receiving metadata information associated with the files stored in the large-scale distributed storage system, and generating a preferred storage parameter for each file based on the received metadata information and the identified file access patterns. The method also includes receiving, file reliability or accessibility information of a new file, determining whether the received file reliability or accessibility information of the new file matches information of a file group of the files in the large-scale distributed storage system, and when the file reliability or accessibility information of the new file matches the information of the file group, storing the new file in the large-scale distributed storage system using the preferred storage parameter associated with the file group.

Abstract translation: 一种方法包括接收表示关于存储在大规模分布式存储系统中的文件的访问信息的跟踪数据，基于跟踪数据识别文件访问模式，接收与存储在大规模分布式存储系统中的文件相关联的元数据信息，以及生成 基于所接收的元数据信息和所识别的文件访问模式，每个文件的优选存储参数。 该方法还包括接收新文件的文件可靠性或可访问性信息，确定新文件的接收文件可靠性或可访问性信息是否匹配大规模分布式存储系统中的文件的文件组的信息，以及当 新文件的文件可靠性或可访问性信息与文件组的信息匹配，使用与文件组相关联的优选存储参数将新文件存储在大规模分布式存储系统中。

公开(公告)号：US09542462B1

公开(公告)日：2017-01-10

申请号：US13918615

申请日：2013-06-14

Applicant: Google Inc.

Inventor： Murray M. Stokely ,  Karl Millar

IPC: G06F17/30

CPC classification number: G06F17/30569 ,  G06F8/443 ,  G06F8/4435 ,  G06F17/30563

Abstract: A system and method for performing large-scale data processing using a statistical programming language are disclosed. One or more high-level statistical operations may be received. The received high-level statistical operations may be dynamically translated into a graph of low-level data operations. The unnecessary operations may be removed and operations may be fused or chained together. Operations may then be grouped into distributed data processing operation. The low-level operations may then be run.

Abstract translation: 公开了一种使用统计编程语言执行大规模数据处理的系统和方法。 可以接收一个或多个高级统计操作。 接收的高级统计操作可以动态地转换成低级数据操作的图形。 可能会删除不必要的操作，并将操作融合或链接在一起。 然后可以将操作分组为分布式数据处理操作。 然后可以运行低级操作。

公开(公告)号：US10339108B2

公开(公告)日：2019-07-02

申请号：US15271739

申请日：2016-09-21

Applicant: Google Inc.

Inventor： Murray M. Stokely ,  Arif Merchant

IPC: G06F17/30 ,  G06F16/17 ,  G06F16/182 ,  G06F3/06

Abstract: A method includes receiving trace data representing access information about files stored in a large-scale distributed storage system, identifying file access patterns based on the trace data, receiving metadata information associated with the files stored in the large-scale distributed storage system, and generating a preferred storage parameter for each file based on the received metadata information and the identified file access patterns. The method also includes receiving, file reliability or accessibility information of a new file, determining whether the received file reliability or accessibility information of the new file matches information of a file group of the files in the large-scale distributed storage system, and when the file reliability or accessibility information of the new file matches the information of the file group, storing the new file in the large-scale distributed storage system using the preferred storage parameter associated with the file group.

公开(公告)号：US09411817B2

公开(公告)日：2016-08-09

申请号：US14034183

申请日：2013-09-23

Applicant: GOOGLE INC.

Inventor： Murray M. Stokely

IPC: G06F17/30

CPC classification number: G06F17/30194

Abstract: Methods to determine and automatically recommend or adjust configuration parameters for storing files in large-scale distributed storage systems are disclosed. These methods may receive file metadata and trace data that allows the system to identify file access patterns. Additionally, the methods may receive information about distributed storage systems in a datacenter. This information can be used to choose storage parameters on a per-file basis for storing files.

Abstract translation: 公开了确定并自动推荐或调整用于在大规模分布式存储系统中存储文件的配置参数的方法。 这些方法可以接收允许系统识别文件访问模式的文件元数据和跟踪数据。 此外，这些方法可以接收关于数据中心中的分布式存储系统的信息。 该信息可用于根据每个文件选择存储参数以存储文件。