公开(公告)号：WO2018106613A1

公开(公告)日：2018-06-14

申请号：PCT/US2017/064558

申请日：2017-12-04

Applicant: GOOGLE LLC

Inventor： ARRIZABALAGA, Javier, Spagnolo ,  NUHN, Malte ,  LE, Quoc, V. ,  DUCKWORTH, Daniel ,  HEILER, Matthias

IPC: G06F17/30

CPC classification number: G06F16/9535 ,  G06F16/95 ,  G06F16/958 ,  G06N3/08

Abstract: Methods, systems, and apparatus including computer programs encoded on a computer storage medium, for augmenting search engine index that indexes resources from a collection of resources. In one aspect, a method of augmenting a search engine index that indexes resources from a collection of resources includes the actions of identifying a resource, in the collection of resources, that is indexed in the search engine index for which a value of a search engine ranking signal is not available; processing data from the resource using a machine learning model, the machine learning model being configured to: process the data to predict a value of the search engine ranking signal for the resource; and updating the search engine index by associating the predicted value of the search engine ranking signal with the resource in the search engine index.

公开(公告)号：WO2022154829A1

公开(公告)日：2022-07-21

申请号：PCT/US2021/043674

申请日：2021-07-29

Applicant: GOOGLE LLC

Inventor： LI, Andrew ,  LI, Sheng ,  TAN, Mingxing ,  PANG, Ruoming ,  CHENG, Liqun ,  LE, Quoc, V. ,  JOUPPI, Norman, Paul

IPC: G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer-readable media, for scaling neural network architectures on hardware accelerators. A method includes receiving training data and information specifying target computing resources, and performing using the training data, a neural architecture search over a search space to identify an architecture for a base neural network. A plurality of scaling parameter values for scaling the base neural network can be identified, which can include repeatedly selecting a plurality of candidate scaling parameter values, and determining a measure of performance for the base neural network scaled according to the plurality of candidate scaling parameter values, in accordance with a plurality of second objectives including a latency objective. An architecture for a scaled neural network can be determined using the architecture of the base neural network scaled according to the plurality of scaling parameter values.

公开(公告)号：WO2021061555A1

公开(公告)日：2021-04-01

申请号：PCT/US2020/051754

申请日：2020-09-21

Applicant: GOOGLE LLC

Inventor： LUONG, Thang, Minh ,  LE, Quoc, V. ,  CLARK, Kevin, Stefan

IPC: G06F40/30 ,  G06N3/08

Abstract: Systems and methods are provided that train a machine-learned language encoding model through the use of a contrastive learning task. In particular, the present disclosure describes a contrastive learning task where the encoder learns to distinguish input tokens from plausible alternatives. In some implementations, on each training example the proposed method masks out some subset (e.g., 15%) of the original input tokens, replaces the masked tokens with samples from a "generator" (e.g., which may be a small masked language model), and then trains the encoder to predict whether each token comes from the original data or is a replacement produced by the generator.