-
公开(公告)号：US11816114B1
公开(公告)日：2023-11-14
申请号：US17533973
申请日：2021-11-23
Applicant: Google LLC
Inventor： Hyung-Jin Kim , Simon Tong , Noam M. Shazeer , Michelangelo Diligenti
IPC: G06F16/2457 , G06F16/951 , G06F16/9535 , G06F16/338 , G06Q30/02
CPC classification number: G06F16/24578 , G06F16/951 , G06F16/9535 , G06F16/338 , G06Q30/02
Abstract: The present disclosure includes systems and techniques relating to ranking search results of a search query. In general, the subject matter described in this specification can be embodied in a computer-implemented method that includes determining a measure of relevance for a document result within a context of a search query for which the document result is returned, the determining being based on a first number in relation to a second number, the first number corresponding to longer views of the document result, and the second number corresponding to at least shorter views of the document result; and outputting the measure of relevance to a ranking engine for ranking of search results, including the document result, for a new search corresponding to the search query. The subject matter described in this specification can also be embodied in various corresponding computer program products, apparatus and systems.
-
公开(公告)号：US20210279576A1
公开(公告)日：2021-09-09
申请号：US17191591
申请日：2021-03-03
Applicant: Google LLC
Inventor： Noam M. Shazeer , Zhenzhong Lan
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing a machine learning task on a network input to generate a network output. In one aspect, one of the systems includes an attention neural network configured to perform the machine learning task, the attention neural network including one or more attention layers, each attention layer comprising an attention sub-layer and, optionally, a feed-forward sub-layer. At least one of the attention layers includes an attention sub-layer that applies talking heads attention instead of conventional multi-head attention.
-
公开(公告)号：US11003993B1
公开(公告)日：2021-05-11
申请号：US16707464
申请日：2019-12-09
Applicant: Google LLC
Inventor： Samy Bengio , Oriol Vinyals , Navdeep Jaitly , Noam M. Shazeer
IPC: G06N3/08
Abstract: This document generally describes a neural network training system, including one or more computers, that trains a recurrent neural network (RNN) to receive an input, e.g., an input sequence, and to generate a sequence of outputs from the input sequence. In some implementations, training can include, for each position after an initial position in a training target sequence, selecting a preceding output of the RNN to provide as input to the RNN at the position, including determining whether to select as the preceding output (i) a true output in a preceding position in the output order or (ii) a value derived from an output of the RNN for the preceding position in an output order generated in accordance with current values of the parameters of the recurrent neural network.
-
公开(公告)号：US20210064924A1
公开(公告)日：2021-03-04
申请号：US17098271
申请日：2020-11-13
Applicant: Google LLC
Inventor： Noam M. Shazeer , Lukasz Mieczyslaw Kaiser , Jakob D. Uszkoreit , Niki J. Parmar , Ashish Teku Vaswani
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating an output image. In one aspect, one of the methods includes generating the output image intensity value by intensity value according to a generation order of pixel—color channel pairs from the output image, comprising, for each particular generation order position in the generation order: generating a current output image representation of a current output image, processing the current output image representation using a decoder neural network to generate a probability distribution over possible intensity values for the pixel—color channel pair at the particular generation order position, wherein the decoder neural network includes one or more local masked self-attention sub-layers; and selecting an intensity value for the pixel—color channel pair at the particular generation order position using the probability distribution.
-
公开(公告)号：US20200342316A1
公开(公告)日：2020-10-29
申请号：US16759690
申请日：2018-10-29
Applicant: GOOGLE LLC
Inventor： Noam M. Shazeer , Lukasz Mieczyslaw Kaiser , Etienne Pot , Mohammad Saleh , Ben David Goodrich , Peter J. Liu , Ryan Sepassi
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating an output sequence from an input sequence. One of the methods includes, at each of a plurality of generation time steps: generating a combined sequence for the generation time step that includes the input sequence followed by the output tokens that have already been generated as of the generation time step; processing the combined sequence using a self-attention decoder neural network to generate a time step output that defines a score distribution over a set of possible output tokens; and selecting, using the time step output, an output token from the set of possible output tokens as the next output token in the output sequence.
-
Patent Agency Ranking
公开(公告)号：US20200082226A1
公开(公告)日：2020-03-12
申请号：US16682611
申请日：2019-11-13
Applicant: Google LLC
Inventor： Noam M. Shazeer , Jakob D. Uszkoreit , Mitchell Thomas Stern
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing parallel generation of output from an autoregressive sequence to sequence model. In one aspect, a blockwise parallel decoding method takes advantage of the fact that some architectures can score sequences in sublinear time. By generating predictions for multiple time steps at once then backing off to a longest prefix validated by the scoring model, the methods can substantially improve the speed of greedy decoding without compromising performance.
-
公开(公告)号：US20190251423A1
公开(公告)日：2019-08-15
申请号：US16393063
申请日：2019-04-24
Applicant: Google LLC
Inventor： Noam M. Shazeer , Azalia Mirhoseini , Krzysztof Stanislaw Maziarz
Abstract: A system includes a neural network that includes a Mixture of Experts (MoE) subnetwork between a first neural network layer and a second neural network layer. The MoE subnetwork includes multiple expert neural networks. Each expert neural network is configured to process a first layer output generated by the first neural network layer to generate a respective expert output. The MoE subnetwork further includes a gating subsystem that selects, based on the first layer output, one or more of the expert neural networks and determine a respective weight for each selected expert neural network, provides the first layer output as input to each of the selected expert neural networks, combines the expert outputs generated by the selected expert neural networks in accordance with the weights for the selected expert neural networks to generate an MoE output, and provides the MoE output as input to the second neural network layer.
-
公开(公告)号：US20190130213A1
公开(公告)日：2019-05-02
申请号：US16174074
申请日：2018-10-29
Applicant: Google LLC
Inventor： Noam M. Shazeer , Lukasz Mieczyslaw Kaiser , Jakob D. Uszkoreit , Niki Parmar , Ashish Teku Vaswani
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating an output image. In one aspect, one of the methods includes generating the output image intensity value by intensity value according to a generation order of pixel-color channel pairs from the output image, comprising, for each particular generation order position in the generation order: generating a current output image representation of a current output image, processing the current output image representation using a decoder neural network to generate a probability distribution over possible intensity values for the pixel-color channel pair at the particular generation order position, wherein the decoder neural network includes one or more local masked self-attention sub-layers; and selecting an intensity value for the pixel-color channel pair at the particular generation order position using the probability distribution.
-
公开(公告)号：US20180341860A1
公开(公告)日：2018-11-29
申请号：US16021971
申请日：2018-06-28
Applicant: Google LLC
Inventor： Noam M. Shazeer , Aidan Nicholas Gomez , Lukasz Mieczyslaw Kaiser , Jakob D. Uszkoreit , Llion Owen Jones , Niki J. Parmar , Illia Polosukhin , Ashish Teku Vaswani
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating an output sequence from an input sequence. In one aspect, one of the systems includes an encoder neural network configured to receive the input sequence and generate encoded representations of the network inputs, the encoder neural network comprising a sequence of one or more encoder subnetworks, each encoder subnetwork configured to receive a respective encoder subnetwork input for each of the input positions and to generate a respective subnetwork output for each of the input positions, and each encoder subnetwork comprising: an encoder self-attention sub-layer that is configured to receive the subnetwork input for each of the input positions and, for each particular input position in the input order: apply an attention mechanism over the encoder subnetwork inputs using one or more queries derived from the encoder subnetwork input at the particular input position.
-
公开(公告)号：US12299573B2
公开(公告)日：2025-05-13
申请号：US18404014
申请日：2024-01-04
Applicant: Google LLC
Inventor： Noam M. Shazeer , Lukasz Mieczyslaw Kaiser , Etienne Pot , Mohammad Saleh , Ben David Goodrich , Peter J. Liu , Ryan Sepassi
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating an output sequence from an input sequence. One of the methods includes, at each of a plurality of generation time steps: generating a combined sequence for the generation time step that includes the input sequence followed by the output tokens that have already been generated as of the generation time step; processing the combined sequence using a self-attention decoder neural network to generate a time step output that defines a score distribution over a set of possible output tokens; and selecting, using the time step output, an output token from the set of possible output tokens as the next output token in the output sequence.
-
-
-
-
-
-
-
-
-
Search Results
76
records
(took 0.024 seconds)
