利索-全球专利检索

  1. 登录
  2. 注册

搜索结果

44 条记录 (耗时 0.022 秒)

    ATTENTION-BASED DECODER-ONLY SEQUENCE TRANSDUCTION NEURAL NETWORKS
    1.
    发明公开
    ATTENTION-BASED DECODER-ONLY SEQUENCE TRANSDUCTION NEURAL NETWORKS 审中-公开

    公开(公告)号：US20240220796A1

    公开(公告)日：2024-07-04

    申请号：US18403992

    申请日：2024-01-04

    申请人： Google LLC

    发明人： Noam M. Shazeer ,  Lukasz Mieczyslaw Kaiser ,  Etienne Pot ,  Mohammad Saleh ,  Ben David Goodrich ,  Peter J. Liu ,  Ryan Sepassi

    IPC分类号： G06N3/08 ,  G06N3/045

    CPC分类号： G06N3/08 ,  G06N3/045

    摘要： Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating an output sequence from an input sequence. One of the methods includes, at each of a plurality of generation time steps: generating a combined sequence for the generation time step that includes the input sequence followed by the output tokens that have already been generated as of the generation time step; processing the combined sequence using a self-attention decoder neural network to generate a time step output that defines a score distribution over a set of possible output tokens; and selecting, using the time step output, an output token from the set of possible output tokens as the next output token in the output sequence.

    Multi-task multi-modal machine learning system
    3.
    发明授权
    Multi-task multi-modal machine learning system 有权

    公开(公告)号：US11494561B2

    公开(公告)日：2022-11-08

    申请号：US16984337

    申请日：2020-08-04

    申请人： Google LLC

    发明人： Noam M. Shazeer ,  Aidan Nicholas Gomez ,  Lukasz Mieczyslaw Kaiser ,  Jakob D. Uszkoreit ,  Llion Owen Jones ,  Niki J. Parmar ,  Ashish Teku Vaswani

    IPC分类号： G06F40/284 ,  G06K9/62 ,  G06N3/04 ,  G06N3/08

    摘要： Methods, systems, and apparatus, including computer programs encoded on computer storage media for training a machine learning model to perform multiple machine learning tasks from multiple machine learning domains. One system includes a machine learning model that includes multiple input modality neural networks corresponding to respective different modalities and being configured to map received data inputs of the corresponding modality to mapped data inputs from a unified representation space; an encoder neural network configured to process mapped data inputs from the unified representation space to generate respective encoder data outputs; a decoder neural network configured to process encoder data outputs to generate respective decoder data outputs from the unified representation space; and multiple output modality neural networks corresponding to respective different modalities and being configured to map decoder data outputs to data outputs of the corresponding modality.

    DEPTHWISE SEPARABLE CONVOLUTIONS FOR NEURAL MACHINE TRANSLATION
    5.
    发明申请
    DEPTHWISE SEPARABLE CONVOLUTIONS FOR NEURAL MACHINE TRANSLATION 有权

    公开(公告)号：US20210073481A1

    公开(公告)日：2021-03-11

    申请号：US17100169

    申请日：2020-11-20

    申请人： Google LLC

    发明人： Aidan Nicholas Gomez ,  Lukasz Mieczyslaw Kaiser ,  Francois Chollet

    IPC分类号： G06F40/58 ,  G06F40/263 ,  G06N3/04 ,  G06N3/08

    摘要： Methods, systems, and apparatus, including computer programs encoded on computer storage media for performing machine translation tasks. One method includes receiving an input text segment in an input language; processing the input text segment using an encoder neural network to generate an encoder neural network output, the encoder neural network comprising multiple depth wise separable convolutional neural network layers; processing the encoder neural network output using an autoregressive decoder neural network to generate a decoder neural network output; and processing the decoder neural network output to generate a predicted output text segment in a target natural language.

    ATTENTION-BASED SEQUENCE TRANSDUCTION NEURAL NETWORKS
    6.
    发明申请
    ATTENTION-BASED SEQUENCE TRANSDUCTION NEURAL NETWORKS 审中-公开

    公开(公告)号：US20190392319A1

    公开(公告)日：2019-12-26

    申请号：US16559392

    申请日：2019-09-03

    申请人： Google LLC

    发明人： Noam M. Shazeer ,  Aidan Nicholas Gomez ,  Lukasz Mieczyslaw Kaiser ,  Jakob D. Uszkoreit ,  Llion Owen Jones ,  Niki J. Parmar ,  Illia Polosukhin ,  Ashish Teku Vaswani

    IPC分类号： G06N3/08 ,  G06N3/04

    摘要： Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating an output sequence from an input sequence. In one aspect, one of the systems includes an encoder neural network configured to receive the input sequence and generate encoded representations of the network inputs, the encoder neural network comprising a sequence of one or more encoder subnetworks, each encoder subnetwork configured to receive a respective encoder subnetwork input for each of the input positions and to generate a respective subnetwork output for each of the input positions, and each encoder subnetwork comprising: an encoder self-attention sub-layer that is configured to receive the subnetwork input for each of the input positions and, for each particular input position in the input order: apply an attention mechanism over the encoder subnetwork inputs using one or more queries derived from the encoder subnetwork input at the particular input position.

    ATTENTION-BASED IMAGE GENERATION NEURAL NETWORKS
    10.
    发明申请
    ATTENTION-BASED IMAGE GENERATION NEURAL NETWORKS 有权

    公开(公告)号：US20230076971A1

    公开(公告)日：2023-03-09

    申请号：US17867242

    申请日：2022-07-18

    申请人： Google LLC

    发明人： Noam M. Shazeer ,  Lukasz Mieczyslaw Kaiser ,  Jakob D. Uszkoreit ,  Niki J. Parmar ,  Ashish Teku Vaswani

    IPC分类号： G06K9/62 ,  G06N3/08 ,  G06N3/04 ,  G06T3/40 ,  G06V10/56

    摘要： Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating an output image. In one aspect, one of the methods includes generating the output image intensity value by intensity value according to a generation order of pixel—color channel pairs from the output image, comprising, for each particular generation order position in the generation order: generating a current output image representation of a current output image, processing the current output image representation using a decoder neural network to generate a probability distribution over possible intensity values for the pixel—color channel pair at the particular generation order position, wherein the decoder neural network includes one or more local masked self-attention sub-layers; and selecting an intensity value for the pixel—color channel pair at the particular generation order position using the probability distribution.