公开(公告)号：US20240282093A1

公开(公告)日：2024-08-22

申请号：US18438317

申请日：2024-02-09

Applicant: Google LLC

Inventor： Alexander Kolesnikov ,  Xiaohua Zhai ,  André Susano Pinto ,  Yuge Shi ,  Lucas Klaus Beyer

IPC: G06V10/82 ,  G06N3/0455

CPC classification number: G06V10/82 ,  G06N3/0455

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for fine-tuning a computer vision neural network. In particular, the neural network is fine-tuned using task rewards and through reinforcement learning.

公开(公告)号：US11983903B2

公开(公告)日：2024-05-14

申请号：US18500034

申请日：2023-11-01

Applicant: Google LLC

Inventor： Neil Matthew Tinmouth Houlsby ,  Sylvain Gelly ,  Jakob D. Uszkoreit ,  Xiaohua Zhai ,  Georg Heigold ,  Lucas Klaus Beyer ,  Alexander Kolesnikov ,  Matthias Johannes Lorenz Minderer ,  Dirk Weissenborn ,  Mostafa Dehghani ,  Alexey Dosovitskiy ,  Thomas Unterthiner

IPC: G06T7/00 ,  G06F18/24 ,  G06N3/045 ,  G06N3/08

CPC classification number: G06T7/97 ,  G06F18/24 ,  G06N3/045 ,  G06N3/08 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing images using self-attention based neural networks. One of the methods includes obtaining one or more images comprising a plurality of pixels; determining, for each image of the one or more images, a plurality of image patches of the image, wherein each image patch comprises a different subset of the pixels of the image; processing, for each image of the one or more images, the corresponding plurality of image patches to generate an input sequence comprising a respective input element at each of a plurality of input positions, wherein a plurality of the input elements correspond to respective different image patches; and processing the input sequences using a neural network to generate a network output that characterizes the one or more images, wherein the neural network comprises one or more self-attention neural network layers.