SAMPLE-EFFICIENT REINFORCEMENT LEARNING

    公开(公告)号:US20210201156A1

    公开(公告)日:2021-07-01

    申请号:US17056640

    申请日:2019-05-20

    Applicant: GOOGLE LLC

    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for sample-efficient reinforcement learning. One of the methods includes maintaining an ensemble of Q networks, an ensemble of transition models, and an ensemble of reward models; obtaining a transition; generating, using the ensemble of transition models, M trajectories; for each time step in each of the trajectories: generating, using the ensemble of reward models, N rewards for the time step, generating, using the ensemble of Q networks, L Q values for the time step, and determining, from the rewards, the Q values, and the training reward, L*N candidate target Q values for the trajectory and for the time step; for each of the time steps, combining the candidate target Q values; determining a final target Q value; and training at least one of the Q networks in the ensemble using the final target Q value.

    Computational graph critical sections

    公开(公告)号:US11188395B2

    公开(公告)日:2021-11-30

    申请号:US16695884

    申请日:2019-11-26

    Applicant: Google LLC

    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for implementing critical section subgraphs in a computational graph system. One of the methods includes executing a lock operation including providing, by a task server, a request to a value server to create a shared critical section object. If the task server determines that the shared critical section object was created by the value server, the task server executes one or more other operations of the critical section subgraph in serial. The task server executes an unlock operation including providing, by the task server, a request to the value server to delete the shared critical section object.

    Database query optimization via parameter-sensitive plan selection

    公开(公告)号:US12235840B2

    公开(公告)日:2025-02-25

    申请号:US18055502

    申请日:2022-11-15

    Applicant: Google LLC

    Abstract: A method includes receiving a database query requesting a database to conditionally return one or more data blocks. The database is stored on memory hardware in communication with the data processing hardware and the database query includes a plurality of parameters characterizing the database query. The method includes generating a set of query plans. Each query plan in the set of query plans is configured to execute the database query using a different order of operations. The method includes training a model using historical database queries and generating, using the trained model, a query plan score for each query plan in the set of query plans. The method includes selecting, using the query plan score of each query plan in the set of query plans, a query plan from the set of query plans. The method also includes executing the database query using the selected query plan.

    Database Query Optimization Via Parameter-Sensitive Plan Selection

    公开(公告)号:US20230153303A1

    公开(公告)日:2023-05-18

    申请号:US18055502

    申请日:2022-11-15

    Applicant: Google LLC

    CPC classification number: G06F16/24542 G06F11/3419

    Abstract: A method includes receiving a database query requesting a database to conditionally return one or more data blocks. The database is stored on memory hardware in communication with the data processing hardware and the database query includes a plurality of parameters characterizing the database query. The method includes generating a set of query plans. Each query plan in the set of query plans is configured to execute the database query using a different order of operations. The method includes training a model using historical database queries and generating, using the trained model, a query plan score for each query plan in the set of query plans. The method includes selecting, using the query plan score of each query plan in the set of query plans, a query plan from the set of query plans. The method also includes executing the database query using the selected query plan.

    Dynamic-length stateful tensor array

    公开(公告)号:US10956500B2

    公开(公告)日:2021-03-23

    申请号:US15410643

    申请日:2017-01-19

    Applicant: Google LLC

    Inventor: Eugene Brevdo

    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for efficiently processing dynamic length tensors of a machine learning model represented by a computational graph. A program is received that specifies a dynamic, iterative computation that can be performed on input data for processing by a machine learning model. A directed computational graph representing the machine learning model is generated that specifies the dynamic, iterative computation as one or more operations using a tensor array object. Input is received for processing by the machine learning model and the directed computational graph representation of the machine learning model is executed with the received input to obtain output.

    COMPUTATIONAL GRAPH CRITICAL SECTIONS
    6.
    发明申请

    公开(公告)号:US20200167207A1

    公开(公告)日:2020-05-28

    申请号:US16695884

    申请日:2019-11-26

    Applicant: Google LLC

    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for implementing critical section subgraphs in a computational graph system. One of the methods includes executing a lock operation including providing, by a task server, a request to a value server to create a shared critical section object. If the task server determines that the shared critical section object was created by the value server, the task server executes one or more other operations of the critical section subgraph in serial. The task server executes an unlock operation including providing, by the task server, a request to the value server to delete the shared critical section object.

    Database Query Optimization Via Parameter-Sensitive Plan Selection

    公开(公告)号:US20250165470A1

    公开(公告)日:2025-05-22

    申请号:US19035387

    申请日:2025-01-23

    Applicant: Google LLC

    Abstract: A method includes receiving a database query requesting a database to conditionally return one or more data blocks. The database is stored on memory hardware in communication with the data processing hardware and the database query includes a plurality of parameters characterizing the database query. The method includes generating a set of query plans. Each query plan in the set of query plans is configured to execute the database query using a different order of operations. The method includes training a model using historical database queries and generating, using the trained model, a query plan score for each query plan in the set of query plans. The method includes selecting, using the query plan score of each query plan in the set of query plans, a query plan from the set of query plans. The method also includes executing the database query using the selected query plan.

    Autonomous Column Selection for Columnar Cache

    公开(公告)号:US20230141891A1

    公开(公告)日:2023-05-11

    申请号:US17523520

    申请日:2021-11-10

    Applicant: Google LLC

    CPC classification number: G06F16/24539 G06F16/2264 G06F16/24552 G06F16/221

    Abstract: Aspects of the disclosure are directed to generating cache configurations for caching data for a database. A database management system (DBMS) can search for column data to cache in a database cache to improve performance of the DBMS in resolving queries. Column data selection can be performed automatically and in the background of a deployed DBMS. Periodically, the DBMS can assess the performance benefit of having certain data cached in the database cache and select data for caching based on the assessed performance benefit. The DBMS can also determine the performance benefit of cached data when not cached, as well as select some portions of data to cache over others. The DBMS can also select data for caching based on different degrees of compression, to further improve query resolution performance.

Patent Agency Ranking