公开(公告)号：US11880340B2

公开(公告)日：2024-01-23

申请号：US16926391

申请日：2020-07-10

Applicant: Google LLC

Inventor： Murray M. Stokely ,  Arif Merchant

IPC: G06F16/17 ,  G06F16/182 ,  G06F3/06

CPC classification number: G06F16/1727 ,  G06F3/067 ,  G06F3/0617 ,  G06F3/0643 ,  G06F16/182 ,  G06F16/184

Abstract: A method includes receiving trace data representing access information about files stored in a large-scale distributed storage system, identifying file access patterns based on the trace data, receiving metadata information associated with the files stored in the large-scale distributed storage system, and generating a preferred storage parameter for each file based on the received metadata information and the identified file access patterns. The method also includes receiving, file reliability or accessibility information of a new file, determining whether the received file reliability or accessibility information of the new file matches information of a file group of the files in the large-scale distributed storage system, and when the file reliability or accessibility information of the new file matches the information of the file group, storing the new file in the large-scale distributed storage system using the preferred storage parameter associated with the file group.

公开(公告)号：US11550712B2

公开(公告)日：2023-01-10

申请号：US16899147

申请日：2020-06-11

Applicant: Google LLC

Inventor： Arif Merchant ,  Lluis Pamies-Juarez

IPC: G06F12/02 ,  G06F9/451 ,  G06N20/00 ,  G06F9/48 ,  G06F9/54 ,  G06N7/00

Abstract: A predictive method for scheduling of the operations is described. The predictive method utilizes data generated from computing an expected lifetime of the individual files or objects within the container. The expected lifetime of individual files or objects can be generated based on machine learning techniques. Operations such as garbage collection are scheduled at an epoch where computational efficiencies are realized for performing the operation.

公开(公告)号：US20250165435A1

公开(公告)日：2025-05-22

申请号：US19033464

申请日：2025-01-21

Applicant: Google LLC

Inventor： Murray M. Stokely ,  Arif Merchant

IPC: G06F16/17 ,  G06F3/06 ,  G06F16/182

Abstract: A method includes receiving trace data representing access information about files stored in a large-scale distributed storage system, identifying file access patterns based on the trace data, receiving metadata information associated with the files stored in the large-scale distributed storage system, and generating a preferred storage parameter for each file based on the received metadata information and the identified file access patterns. The method also includes receiving, file reliability or accessibility information of a new file, determining whether the received file reliability or accessibility information of the new file matches information of a file group of the files in the large-scale distributed storage system, and when the file reliability or accessibility information of the new file matches the information of the file group, storing the new file in the large-scale distributed storage system using the preferred storage parameter associated with the file group.

公开(公告)号：US12222901B2

公开(公告)日：2025-02-11

申请号：US17445339

申请日：2021-08-18

Applicant: Google LLC

Inventor： Murray M. Stokely ,  Arif Merchant

IPC: G06F16/17 ,  G06F3/06 ,  G06F16/182

Abstract: A method includes receiving trace data representing access information about files stored in a large-scale distributed storage system, identifying file access patterns based on the trace data, receiving metadata information associated with the files stored in the large-scale distributed storage system, and generating a preferred storage parameter for each file based on the received metadata information and the identified file access patterns. The method also includes receiving, file reliability or accessibility information of a new file, determining whether the received file reliability or accessibility information of the new file matches information of a file group of the files in the large-scale distributed storage system, and when the file reliability or accessibility information of the new file matches the information of the file group, storing the new file in the large-scale distributed storage system using the preferred storage parameter associated with the file group.

公开(公告)号：US11442937B2

公开(公告)日：2022-09-13

申请号：US16899180

申请日：2020-06-11

Applicant: Google LLC

Inventor： Arif Merchant ,  Tzu-Wei Yang ,  Mustafa Uysal

IPC: G06F16/2455 ,  G06F16/2453 ,  G06F16/22 ,  G06F12/02 ,  G06F9/30 ,  G06F12/0813

Abstract: The technology is directed to cache admission control. One or more processors may categorize access requests for data items in a cache storage into a plurality of categories and collect information on the access requests over time. Based on the collected information, a utility value for caching data items in each category of the plurality of categories may be determined. Newly requested data items may be admitted into the cache storage in an order according to the corresponding utility values of their respective categories.

公开(公告)号：US20210390045A1

公开(公告)日：2021-12-16

申请号：US16899147

申请日：2020-06-11

Applicant: Google LLC

Inventor： Arif Merchant ,  Lluis Pamies-Juarez

IPC: G06F12/02 ,  G06F9/54 ,  G06F9/451 ,  G06F9/48 ,  G06N20/00 ,  G06N7/00

Abstract: A predictive method for scheduling of the operations is described. The predictive method utilizes data generated from computing an expected lifetime of the individual files or objects within the container. The expected lifetime of individual files or objects can be generated based on machine learning techniques. Operations such as garbage collection are scheduled at an epoch where computational efficiencies are realized for performing the operation.

公开(公告)号：US11113242B2

公开(公告)日：2021-09-07

申请号：US16434376

申请日：2019-06-07

Applicant: Google LLC

Inventor： Murray M. Stokely ,  Arif Merchant

IPC: G06F16/17 ,  G06F16/182 ,  G06F3/06

Abstract: A method includes receiving trace data representing access information about files stored in a large-scale distributed storage system, identifying file access patterns based on the trace data, receiving metadata information associated with the files stored in the large-scale distributed storage system, and generating a preferred storage parameter for each file based on the received metadata information and the identified file access patterns. The method also includes receiving, file reliability or accessibility information of a new file, determining whether the received file reliability or accessibility information of the new file matches information of a file group of the files in the large-scale distributed storage system, and when the file reliability or accessibility information of the new file matches the information of the file group, storing the new file in the large-scale distributed storage system using the preferred storage parameter associated with the file group.

公开(公告)号：US11995037B2

公开(公告)日：2024-05-28

申请号：US16702535

申请日：2019-12-03

Applicant: Google LLC

Inventor： Murray M. Stokely ,  Arif Merchant

IPC: G06F16/17 ,  G06F3/06 ,  G06F16/182

CPC classification number: G06F16/1727 ,  G06F3/0617 ,  G06F3/0643 ,  G06F3/067 ,  G06F16/182 ,  G06F16/184

Abstract: A method includes receiving trace data representing access information about files stored in a large-scale distributed storage system, identifying file access patterns based on the trace data, receiving metadata information associated with the files stored in the large-scale distributed storage system, and generating a preferred storage parameter for each file based on the received metadata information and the identified file access patterns. The method also includes receiving, file reliability or accessibility information of a new file, determining whether the received file reliability or accessibility information of the new file matches information of a file group of the files in the large-scale distributed storage system, and when the file reliability or accessibility information of the new file matches the information of the file group, storing the new file in the large-scale distributed storage system using the preferred storage parameter associated with the file group.

公开(公告)号：US20220272148A1

公开(公告)日：2022-08-25

申请号：US17663618

申请日：2022-05-16

Applicant: Google LLC

Inventor： Alexander Shraer ,  Kfir Lev-Ari ,  Arif Merchant ,  Vishesh Khemani ,  Atul Adya

IPC: H04L67/1001 ,  G06F9/50 ,  G06F16/00 ,  G06F16/27 ,  H04L43/08 ,  H04L67/148

Abstract: Methods, systems, and apparatus for automatic sharding and load balancing in a distributed data processing system. In one aspect, a method includes determining workload distribution for an application across worker computers and in response to determining a load balancing operation is required: selecting a first worker computer having a highest load measure relative to respective load measure of the other work computers; determining one or more move operations for a partition of data assigned to the first worker computer and a weight for each move operation; and selecting the move operation with a highest weight the selected move operation.

公开(公告)号：US20210382852A1

公开(公告)日：2021-12-09

申请号：US17445339

申请日：2021-08-18

Applicant: Google LLC

Inventor： Murray M. Stokely ,  Arif Merchant

IPC: G06F16/17 ,  G06F16/182 ,  G06F3/06

Abstract: A method includes receiving trace data representing access information about files stored in a large-scale distributed storage system, identifying file access patterns based on the trace data, receiving metadata information associated with the files stored in the large-scale distributed storage system, and generating a preferred storage parameter for each file based on the received metadata information and the identified file access patterns. The method also includes receiving, file reliability or accessibility information of a new file, determining whether the received file reliability or accessibility information of the new file matches information of a file group of the files in the large-scale distributed storage system, and when the file reliability or accessibility information of the new file matches the information of the file group, storing the new file in the large-scale distributed storage system using the preferred storage parameter associated with the file group.