中科微点-全球专利检索

  1. Login
  2. Sign Up

Search Results

13 records (took 0.02 seconds)

    ENERGY BASED PROCESSES FOR EXCHANGEABLE DATA
    1.
    发明申请
    ENERGY BASED PROCESSES FOR EXCHANGEABLE DATA 有权

    公开(公告)号：US20220343152A1

    公开(公告)日：2022-10-27

    申请号：US17239320

    申请日：2021-04-23

    Applicant: Google LLC

    Inventor： Bo Dai ,  Mengjiao Yang ,  Hanjun Dai ,  Dale Eric Schuurmans

    IPC: G06N3/08 ,  G06N3/04

    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generative modelling of an exchangeable sets. Methods can include obtaining a dataset of training observations. Each training observation is an exchangeable set that includes a plurality of data points. Each training observations is processed using a first neural network to generate parameters of a first probability distribution based on which a latent variable is sampled. The latent variable is processed using a second neural network to generate a new observation that includes a plurality of data points. The training observation and the new observation is processed using an energy neural network to generate an estimate of an energy of the training observation and the new observation. The energy neural network is then trained to optimize an objective function that measures the difference between the estimate of the energy of the training observation and the new observation.

    Full Attention with Sparse Computation Cost
    4.
    发明申请
    Full Attention with Sparse Computation Cost 有权

    公开(公告)号：US20230022151A1

    公开(公告)日：2023-01-26

    申请号：US17860691

    申请日：2022-07-08

    Applicant: Google LLC

    Inventor： Hanjun Dai ,  Bo Dai ,  Hongyu Ren ,  Dale Eric Schuurmans ,  Zihang Dai ,  Mengjiao Yang

    IPC: G06F40/20 ,  G06N20/00

    Abstract: The present disclosure is directed to machine learning model architectures which provide full attention capability in each attention head while maintaining low computation and memory complexity. Specifically, according to one aspect of the present disclosure, example attention models provided herein can treat the self-attention mechanism as a conditional expectation over embeddings at each location and approximate the conditional distribution with a structured factorization. Each location can attend to all other locations, either via direct attention, or through indirect attention to group representations, which are again conditional expectations of embeddings from corresponding local regions.

    Universal Self-Adaptive Prompting
    5.
    发明申请
    Universal Self-Adaptive Prompting 有权

    公开(公告)号：US20240394545A1

    公开(公告)日：2024-11-28

    申请号：US18377368

    申请日：2023-10-06

    Applicant: Google LLC

    Inventor： Julian Martin Eisenschlos ,  Xingchen Wan ,  Hootan Nakhost ,  Sercan Omer Arik ,  Ruoxi Sun ,  Hanjun Dai

    IPC: G06N3/088 ,  G06N3/0455

    Abstract: Aspects of the disclosure are directed to methods, systems, and computer readable media for universal self-adaptive prompting (USP), which includes an automatic prompt design approach specifically tailored for zero-shot learning, though still compatible with few-shot learning. To achieve universal prompting, USP categorizes a natural language processing (NLP) task into one of a plurality of possible task types and then uses a corresponding selector to select the most suitable queries and zero-shot model-generated responses as pseudo-demonstrations, thereby generalizing in-context learning to the zero-shot setup in a fully automated manner.

    GRADIENT-FREE STRUCTURED PRUNING OF NEURAL NETWORKS
    7.
    发明公开
    GRADIENT-FREE STRUCTURED PRUNING OF NEURAL NETWORKS 审中-公开

    公开(公告)号：US20240289619A1

    公开(公告)日：2024-08-29

    申请号：US18424595

    申请日：2024-01-26

    Applicant: Google LLC

    Inventor： Azade Nova ,  Hanjun Dai ,  Dale Eric Schuurmans

    IPC: G06N3/082 ,  G06N3/048

    CPC classification number: G06N3/082 ,  G06N3/048

    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing a machine learning task on a network input to generate a network output. One of the methods includes: obtaining data specifying an initial neural network configured to perform a machine learning task; a representativeness measure for each of a plurality of filters; determining a central tendency measure for the plurality of filters based on processing a batch of network inputs using the initial neural network; determining a cumulative importance score for each of the plurality of filters; selecting a proper subset of the plurality of filters; and generating a pruned neural network configured to perform the machine learning task.

    AUTOREGRESSIVE GRAPH GENERATION MACHINE LEARNING MODELS
    8.
    发明申请
    AUTOREGRESSIVE GRAPH GENERATION MACHINE LEARNING MODELS 有权

    公开(公告)号：US20220414067A1

    公开(公告)日：2022-12-29

    申请号：US17351086

    申请日：2021-06-17

    Applicant: Google LLC

    Inventor： Hanjun Dai ,  Azade Nazi ,  Yujia Li ,  Bo Dai ,  Dale Eric Schuurmans

    IPC: G06F16/21 ,  G06N3/04 ,  G06F16/22

    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating data defining a graph. In one aspect, a method comprises: sequentially generating a respective edge set for each node in the graph, wherein for each of a plurality of nodes after a first node, generating the edge set for the node comprises: receiving a context embedding for the node that summarizes a respective edge set for each node that precedes the node; generating, based on the context embedding for the node: (i) a respective edge set for the node, and (ii) a respective embedding of the edge set for the node; generating a context embedding for a next node in the ordering of the nodes using the embedding of the edge set for the node; and adding the set of edges defined by the edge set for the node to the graph.

    STOCHASTIC OPTIMIZATION USING MACHINE LEARNING
    9.
    发明申请
    STOCHASTIC OPTIMIZATION USING MACHINE LEARNING 有权

    公开(公告)号：US20250045577A1

    公开(公告)日：2025-02-06

    申请号：US18697304

    申请日：2021-10-05

    Applicant: Google LLC

    Inventor： Bo Dai ,  Hanjun Dai ,  Yuan Xue ,  Zia Syed ,  Dale Eric Schuurmans

    IPC: G06N3/08

    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing stochastic optimization using machine learning. One of the methods includes obtaining data defining a multi-stage stochastic optimization (MSSO) problem instance, the data characterizing an observation distribution, an action space, and a cost function; generating a neural network input characterizing the MSSO problem instance from the data; providing the neural network input as input to a neural network that generates, from the network input, a neural network output characterizing parameters of a value function corresponding to the MSSO problem instance; processing the neural network input using the neural network to generate the neural network output; obtaining a new observation determined according to the observation distribution for the MSSO problem instance; determining, using the value function characterized by the network output, an optimal action to take in response to the new observation; and executing the optimal action.

Patent Agency Ranking