公开(公告)号：US09239794B1

公开(公告)日：2016-01-19

申请号：US13873682

申请日：2013-04-30

Applicant: Google Inc.

Inventor： Arif Merchant

IPC: G06F12/00 ,  G06F12/08

CPC classification number: G06F12/0862 ,  G06F12/0868 ,  G06F2212/1016 ,  G06F2212/6026

Abstract: A method for managing data items retrieved for storage in a prefetch memory buffer includes determining a probability that a first data item will be requested for retrieval. The method includes estimating a first request time at which the new data item will be requested. The method also includes determining a time differential for the first data item, wherein the time differential is determined based on current time and the first request time. The method includes calculating a first prefetch priority value for the first data item based on the first data item probability and the time differential. The method includes randomly comparing the first prefetch priority value of the first data item to the prefetch priority values of the one or more stored data items to identify at least one stored data item having a prefetch priority value lower than the first prefetch priority value.

Abstract translation: 用于管理检索用于存储在预取存储器缓冲器中的数据项的方法包括确定将要请求第一数据项进行检索的概率。 该方法包括估计将请求新数据项的第一请求时间。 该方法还包括确定第一数据项的时间差，其中基于当前时间和第一请求时间来确定时间差。 该方法包括基于第一数据项概率和时间差计算第一数据项的第一预取优先级值。 该方法包括将第一数据项的第一预取优先级值与一个或多个存储数据项的预取优先级值进行随机比较，以识别具有低于第一预取优先级值的预取优先级值的至少一个存储数据项。

公开(公告)号：US09128825B1

公开(公告)日：2015-09-08

申请号：US13897159

申请日：2013-05-17

Applicant: GOOGLE INC.

Inventor： Christoph Albrecht ,  Murray Stokely ,  Arif Merchant ,  Christian Eric Schrock ,  Xudong Shi

IPC: G06F12/02

CPC classification number: G06F12/0246 ,  G06F3/0608 ,  G06F3/061 ,  G06F3/0649 ,  G06F3/0665 ,  G06F3/0685 ,  G06F12/02 ,  G06F12/0253

Abstract: Systems and methods are discussed relating to allocation of memory from a fixed pool of fast memory within a data center having a data storage area equipped with that memory. Techniques include: receiving a request to write data in the storage area; identifying a file group associated with the write request; analyzing previous data activity traces associated with the file group; determining an available fast memory amount based on the total amount of fast memory in the fixed pool and a currently allocated amount of fast memory; determining a fast memory allocation for the file group based on the previous data activity traces, the available fast memory, and a fast memory constraint, the memory allocation including an allocation amount and a write probability; and providing information about the memory allocation to a file system of the data center, which writes the data based on the allocation amount and write probability.

Abstract translation: 讨论了关于在具有配备有该存储器的数据存储区域的数据中心内从快速存储器的固定池分配存储器的系统和方法。 技术包括：接收在存储区域中写入数据的请求; 识别与所述写请求相关联的文件组; 分析与文件组相关联的以前的数据活动跟踪; 基于固定池中的快速存储器的总量和当前分配的快速存储量来确定可用的快速存储器量; 基于先前的数据活动轨迹，可用的快速存储器和快速存储器约束确定文件组的快速存储器分配，存储器分配包括分配量和写入概率; 并且向数据中心的文件系统提供关于存储器分配的信息，其基于分配量和写入概率来写入数据。

公开(公告)号：US09043512B1

公开(公告)日：2015-05-26

申请号：US14591733

申请日：2015-01-07

Applicant: Google Inc.

Inventor： Arif Merchant

IPC: G06F3/00 ,  G06F13/16 ,  G06F3/06

CPC classification number: G06F13/1642 ,  G06F3/0611 ,  G06F3/0659 ,  G06F3/0683 ,  G06F13/1626

Abstract: Systems, mediums, and methods are provided for scheduling input/output requests to a storage system. The input output requests may be received, categorized based on their priority, and scheduled for retrieval from the storage system. Lower priority requests may be divided into smaller sub-requests, and the sub-requests may be scheduled for retrieval only when there are no pending higher priority requests, and/or when higher priority requests are not predicted to arrive for a certain period of time. By servicing the small sub-requests rather than the entire lower priority request, the retrieval of the lower priority request may be paused in the event that a high priority request arrives while the lower priority request is being serviced.

Abstract translation: 提供了系统，介质和方法，用于将输入/输出请求调度到存储系统。 可以接收输入输出请求，基于其优先级进行分类，并且计划从存储系统检索。 低优先级请求可以被分成较小的子请求，并且只有当没有未决的更高优先级的请求时，和/或当较高优先级的请求不被预测到达某个时间段时，可以将子请求调度为进行检索 。 通过对小的子请求进行服务而不是整个较低优先级的请求，在优先级较低的请求被服务时，在优先级高的请求到达的情况下，低优先权请求的检索可以被暂停。

公开(公告)号：US20170011059A1

公开(公告)日：2017-01-12

申请号：US15271739

申请日：2016-09-21

Applicant: Google Inc.

Inventor： Murray M. Stokely ,  Arif Merchant

IPC: G06F17/30 ,  G06F3/06

CPC classification number: G06F16/1727 ,  G06F3/0617 ,  G06F3/0643 ,  G06F3/067 ,  G06F16/182 ,  G06F16/184

Abstract: A method includes receiving trace data representing access information about files stored in a large-scale distributed storage system, identifying file access patterns based on the trace data, receiving metadata information associated with the files stored in the large-scale distributed storage system, and generating a preferred storage parameter for each file based on the received metadata information and the identified file access patterns. The method also includes receiving, file reliability or accessibility information of a new file, determining whether the received file reliability or accessibility information of the new file matches information of a file group of the files in the large-scale distributed storage system, and when the file reliability or accessibility information of the new file matches the information of the file group, storing the new file in the large-scale distributed storage system using the preferred storage parameter associated with the file group.

Abstract translation: 一种方法包括接收表示关于存储在大规模分布式存储系统中的文件的访问信息的跟踪数据，基于跟踪数据识别文件访问模式，接收与存储在大规模分布式存储系统中的文件相关联的元数据信息，以及生成 基于所接收的元数据信息和所识别的文件访问模式，每个文件的优选存储参数。 该方法还包括接收新文件的文件可靠性或可访问性信息，确定新文件的接收文件可靠性或可访问性信息是否匹配大规模分布式存储系统中的文件的文件组的信息，以及当 新文件的文件可靠性或可访问性信息与文件组的信息匹配，使用与文件组相关联的优选存储参数将新文件存储在大规模分布式存储系统中。

公开(公告)号：US09477679B2

公开(公告)日：2016-10-25

申请号：US14033255

申请日：2013-09-20

Applicant: GOOGLE INC.

Inventor： Murray M. Stokely ,  Arif Merchant

IPC: G06F17/30

CPC classification number: G06F17/30138 ,  G06F3/0617 ,  G06F3/0643 ,  G06F3/067 ,  G06F17/30194 ,  G06F17/30212

Abstract: Methods to determine and automatically recommend or adjust configuration parameters for storing files in large-scale distributed storage systems are disclosed. These methods may receive file metadata and trace data that allows the system to identify file access patterns. Additionally, the methods may receive information about distributed storage systems in a datacenter. This information can be used to choose storage parameters on a per-file basis for storing files.

Abstract translation: 公开了确定并自动推荐或调整用于在大规模分布式存储系统中存储文件的配置参数的方法。 这些方法可以接收允许系统识别文件访问模式的文件元数据和跟踪数据。 此外，这些方法可以接收关于数据中心中的分布式存储系统的信息。 该信息可用于根据每个文件选择存储参数以存储文件。

公开(公告)号：US10339108B2

公开(公告)日：2019-07-02

申请号：US15271739

申请日：2016-09-21

Applicant: Google Inc.

Inventor： Murray M. Stokely ,  Arif Merchant

IPC: G06F17/30 ,  G06F16/17 ,  G06F16/182 ,  G06F3/06

Abstract: A method includes receiving trace data representing access information about files stored in a large-scale distributed storage system, identifying file access patterns based on the trace data, receiving metadata information associated with the files stored in the large-scale distributed storage system, and generating a preferred storage parameter for each file based on the received metadata information and the identified file access patterns. The method also includes receiving, file reliability or accessibility information of a new file, determining whether the received file reliability or accessibility information of the new file matches information of a file group of the files in the large-scale distributed storage system, and when the file reliability or accessibility information of the new file matches the information of the file group, storing the new file in the large-scale distributed storage system using the preferred storage parameter associated with the file group.