公开(公告)号：US11775830B2

公开(公告)日：2023-10-03

申请号：US18079791

申请日：2022-12-12

Applicant: DeepMind Technologies Limited

Inventor： Chongli Qin ,  Sven Adrian Gowal ,  Soham De ,  Robert Stanforth ,  James Martens ,  Krishnamurthy Dvijotham ,  Dilip Krishnan ,  Alhussein Fawzi

IPC: G06N3/08 ,  G06V10/82 ,  G06F18/214 ,  G06F18/2135 ,  G06V10/764 ,  G06V10/774

CPC classification number: G06N3/08 ,  G06F18/214 ,  G06F18/21355 ,  G06V10/764 ,  G06V10/774 ,  G06V10/82

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network. One of the methods includes processing each training input using the neural network and in accordance with the current values of the network parameters to generate a network output for the training input; computing a respective loss for each of the training inputs by evaluating a loss function; identifying, from a plurality of possible perturbations, a maximally non-linear perturbation; and determining an update to the current values of the parameters of the neural network by performing an iteration of a neural network training procedure to decrease the respective losses for the training inputs and to decrease the non-linearity of the loss function for the identified maximally non-linear perturbation.

公开(公告)号：US20230252286A1

公开(公告)日：2023-08-10

申请号：US18079791

申请日：2022-12-12

Applicant: DeepMind Technologies Limited

Inventor： Chongli Qin ,  Sven Adrian Gowal ,  Soham De ,  Robert Stanforth ,  James Martens ,  Krishnamurthy Dvijotham ,  Dilip Krishnan ,  Alhussein Fawzi

IPC: G06N3/08 ,  G06V10/82 ,  G06F18/214 ,  G06F18/2135 ,  G06V10/764 ,  G06V10/774

CPC classification number: G06N3/08 ,  G06V10/82 ,  G06F18/214 ,  G06F18/21355 ,  G06V10/764 ,  G06V10/774

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network. One of the methods includes processing each training input using the neural network and in accordance with the current values of the network parameters to generate a network output for the training input; computing a respective loss for each of the training inputs by evaluating a loss function; identifying, from a plurality of possible perturbations, a maximally non-linear perturbation; and determining an update to the current values of the parameters of the neural network by performing an iteration of a neural network training procedure to decrease the respective losses for the training inputs and to decrease the non-linearity of the loss function for the identified maximally non-linear perturbation.

公开(公告)号：US20210149968A1

公开(公告)日：2021-05-20

申请号：US16951920

申请日：2020-11-18

Applicant: DeepMind Technologies Limited

Inventor： Anton Zhernov ,  Krishnamurthy Dvijotham ,  Xiaohong Gong ,  Amogh S. Asgekar

IPC: G06F16/93 ,  G06N20/00 ,  G06N5/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for re-ranking a collection of documents according to a first metric and subject to a constraint on a function of one or more second metrics. One of the methods includes: obtaining, for each document in the first collection of documents, a respective first metric value corresponding to the first metric and respective one or more second metric values corresponding to the one or more second metrics; re-ranking the first collection of documents, comprising: determining the constraint on the function of one or more second metrics by computing a first threshold value using a variable threshold function that takes as input second metric values for the documents in the first collection of documents; and determining the re-ranking for the first collection of documents by solving a constrained optimization for the first metric constrained by the first threshold value.

公开(公告)号：US12001484B2

公开(公告)日：2024-06-04

申请号：US17177097

申请日：2021-02-16

Applicant: DeepMind Technologies Limited

Inventor： Timothy Arthur Mann ,  Ivan Lobov ,  Anton Zhernov ,  Krishnamurthy Dvijotham ,  Xiaohong Gong ,  Dan-Andrei Calian

IPC: G06F16/95 ,  G06F16/903 ,  G06F17/11 ,  G06F17/16

CPC classification number: G06F16/90335 ,  G06F17/11 ,  G06F17/16

Abstract: Methods and systems for low-latency multi-constraint ranking of content items. One of the methods includes receiving a request to rank a plurality of content items for presentation to a user to maximize a primary objective subject to a plurality of constraints; initializing a dual variable vector; updating the dual variable vector, comprising: determining an overall objective score for the dual variable vector; identifying a plurality of candidate dual variable vectors that includes one or more neighboring node dual variable vectors; determining respective overall objective scores for each of the one or more candidate dual variable vectors; identifying the candidate with the best overall objective score; and determining whether to update the dual variable vector based on whether the identified candidate has a better overall objective score than the dual variable vector; and determining a final ranking for the content items based on the dual variable vector.

公开(公告)号：US11847414B2

公开(公告)日：2023-12-19

申请号：US17239284

申请日：2021-04-23

Applicant: DeepMind Technologies Limited

Inventor： Krishnamurthy Dvijotham ,  Anton Zhernov ,  Sven Adrian Gowal ,  Conrad Grobler ,  Robert Stanforth

IPC: G06F40/279 ,  G06F40/247 ,  G06N5/04 ,  G06N20/00 ,  G06F40/166

CPC classification number: G06F40/279 ,  G06F40/166 ,  G06F40/247 ,  G06N5/04 ,  G06N20/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a text classification machine learning model. One of the methods includes training a model having a plurality of parameters and configured to generate a classification of a text sample comprising a plurality of words by processing a model input that includes a combined feature representation of the plurality of words in the text sample, wherein the training comprises receiving a text sample and a target classification for the text sample; generating a plurality of perturbed combined feature representations; determining, based on the plurality of perturbed combined feature representations, a region in the embedding space; and determining an update to the parameters based on an adversarial objective that encourages the model to assign the target classification for the text sample for all of the combined feature representations in the region in the embedding space.

公开(公告)号：US11526755B2

公开(公告)日：2022-12-13

申请号：US16882332

申请日：2020-05-22

Applicant: DeepMind Technologies Limited

Inventor： Chongli Qin ,  Sven Adrian Gowal ,  Soham De ,  Robert Stanforth ,  James Martens ,  Krishnamurthy Dvijotham ,  Dilip Krishnan ,  Alhussein Fawzi

IPC: G06N3/08 ,  G06K9/62 ,  G06V10/82

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network. One of the methods includes processing each training input using the neural network and in accordance with the current values of the network parameters to generate a network output for the training input; computing a respective loss for each of the training inputs by evaluating a loss function; identifying, from a plurality of possible perturbations, a maximally non-linear perturbation; and determining an update to the current values of the parameters of the neural network by performing an iteration of a neural network training procedure to decrease the respective losses for the training inputs and to decrease the non-linearity of the loss function for the identified maximally non-linear perturbation.

公开(公告)号：US20240143696A1

公开(公告)日：2024-05-02

申请号：US18275737

申请日：2022-02-07

Applicant: DeepMind Technologies Limited

Inventor： Ali Taylan Cemgil ,  Krishnamurthy Dvijotham ,  Arnaud Doucet ,  Jamie Hayes

IPC: G06F17/18

CPC classification number: G06F17/18

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating one or more differentiable order statistics for a vector of scores. In one aspect, a method comprises: obtaining the vector of scores, wherein each position in the vector of scores is associated with a respective index from a set of indices; obtaining a plurality of pairs of indices; generating a respective swapping probability for each pair of indices based on the vector of scores; generating, for each pair of indices, a respective soft-swapping matrix for the pair of indices as a combination of: (i) an identity matrix, and (ii) an exchange matrix, wherein the exchange matrix is weighted in the combination by the swapping probability for the pair of indices; and generating the one or more differentiable order statistics for the vector of scores using the soft-swapping matrices.

公开(公告)号：US11675855B2

公开(公告)日：2023-06-13

申请号：US16951920

申请日：2020-11-18

Applicant: DeepMind Technologies Limited

Inventor： Anton Zhernov ,  Krishnamurthy Dvijotham ,  Xiaohong Gong ,  Amogh S. Asgekar

IPC: G06F17/00 ,  G06F16/93 ,  G06N5/04 ,  G06N20/00

CPC classification number: G06F16/93 ,  G06N5/04 ,  G06N20/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for re-ranking a collection of documents according to a first metric and subject to a constraint on a function of one or more second metrics. One of the methods includes: obtaining, for each document in the first collection of documents, a respective first metric value corresponding to the first metric and respective one or more second metric values corresponding to the one or more second metrics; re-ranking the first collection of documents, comprising: determining the constraint on the function of one or more second metrics by computing a first threshold value using a variable threshold function that takes as input second metric values for the documents in the first collection of documents; and determining the re-ranking for the first collection of documents by solving a constrained optimization for the first metric constrained by the first threshold value.

公开(公告)号：US20210334459A1

公开(公告)日：2021-10-28

申请号：US17239284

申请日：2021-04-23

Applicant: DeepMind Technologies Limited

Inventor： Krishnamurthy Dvijotham ,  Anton Zhernov ,  Sven Adrian Gowal ,  Conrad Grobler ,  Robert Stanforth

IPC: G06F40/279 ,  G06F40/247 ,  G06F40/166 ,  G06N20/00 ,  G06N5/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a text classification machine learning model. One of the methods includes training a model having a plurality of parameters and configured to generate a classification of a text sample comprising a plurality of words by processing a model input that includes a combined feature representation of the plurality of words in the text sample, wherein the training comprises receiving a text sample and a target classification for the text sample; generating a plurality of perturbed combined feature representations; determining, based on the plurality of perturbed combined feature representations, a region in the embedding space; and determining an update to the parameters based on an adversarial objective that encourages the model to assign the target classification for the text sample for all of the combined feature representations in the region in the embedding space.

公开(公告)号：US20210256072A1

公开(公告)日：2021-08-19

申请号：US17177097

申请日：2021-02-16

Applicant: DeepMind Technologies Limited

Inventor： Timothy Arthur Mann ,  Ivan Lobov ,  Anton Zhernov ,  Krishnamurthy Dvijotham ,  Xiaohong Gong ,  Dan-Andrei Calian

IPC: G06F16/903 ,  G06F17/16 ,  G06F17/11

Abstract: Methods and systems for low-latency multi-constraint ranking of content items. One of the methods includes receiving a request to rank a plurality of content items for presentation to a user to maximize a primary objective subject to a plurality of constraints; initializing a dual variable vector; updating the dual variable vector, comprising: determining an overall objective score for the dual variable vector; identifying a plurality of candidate dual variable vectors that includes one or more neighboring node dual variable vectors; determining respective overall objective scores for each of the one or more candidate dual variable vectors; identifying the candidate with the best overall objective score; and determining whether to update the dual variable vector based on whether the identified candidate has a better overall objective score than the dual variable vector; and determining a final ranking for the content items based on the dual variable vector.