公开(公告)号：US20210209475A1

公开(公告)日：2021-07-08

申请号：US17056549

申请日：2019-05-20

Applicant: Google LLC

Inventor： Shumeet Baluja ,  David Marwood

IPC: G06N3/08 ,  G06N3/04

Abstract: The present disclosure provides systems and methods that train and use neural networks that can be run with no multiplications and no floating point operations. In particular, according to one aspect of the present disclosure, the respective non-linear and continuous activation functions typically used by the nodes of a neural network can be replaced with custom activation functions that output one of a discrete number of activation values. Likewise, according to another aspect of the present disclosure, the neural network can be trained such that each of its weights equals one of a discrete number of weight values. Taken together, this enables replacement of the typical multiplication process associated with computing a node of the network with a simple, and much faster, lookup process. In particular, a lookup table can store the result of multiplying each unique pair of activation value and weight value.

公开(公告)号：US20240370706A1

公开(公告)日：2024-11-07

申请号：US18690176

申请日：2021-10-01

Applicant: Google LLC

Inventor： David Marwood ,  Shumeet Baluja

IPC: G06N3/0464

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing an input through each of a plurality of layers of a neural network to generate an output, wherein the plurality of layers comprise a convolutional layer. One of the methods includes: receiving a layer input for the convolutional layer; processing the layer input to generate a layer output for the convolutional layer, comprising determining a convolution between the layer input and a filter associated with the convolutional layer; generating a spatial weight mask for the convolutional layer by using a contextual convolution block in accordance with a set of one or more spatially sensitive mask functions defined in the contextual convolution block; and determining a weighted layer output for the convolutional layer, comprising determining a product between the spatial weight mask and the layer output of the convolutional layer.

公开(公告)号：US11488016B2

公开(公告)日：2022-11-01

申请号：US16751175

申请日：2020-01-23

Applicant: Google LLC

Inventor： Michele Covell ,  David Marwood ,  Shumeet Baluja ,  Nicholas Johnston

IPC: G06N3/04 ,  G06N3/08 ,  G06F1/03

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing a network input using a neural network to generate a network output for the network input. One of the methods includes maintaining, for each of the plurality of neural network layers, a respective look-up table that maps each possible combination of a quantized input index and a quantized weight index to a multiplication result; and generating a network output from a network input, comprising, for each of the neural network layers: receiving data specifying a quantized input to the neural network layer, the quantized input comprising a plurality of quantized input values; and generating a layer output for the neural network layer from the quantized input to the neural network layer using the respective look-up table for the neural network layer.

公开(公告)号：US20200234126A1

公开(公告)日：2020-07-23

申请号：US16751175

申请日：2020-01-23

Applicant: Google LLC

Inventor： Michele Covell ,  David Marwood ,  Shumeet Baluja ,  Nicholas Johnston

IPC: G06N3/08 ,  G06N3/04 ,  G06F1/03

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing a network input using a neural network to generate a network output for the network input. One of the methods includes maintaining, for each of the plurality of neural network layers, a respective look-up table that maps each possible combination of a quantized input index and a quantized weight index to a multiplication result; and generating a network output from a network input, comprising, for each of the neural network layers: receiving data specifying a quantized input to the neural network layer, the quantized input comprising a plurality of quantized input values; and generating a layer output for the neural network layer from the quantized input to the neural network layer using the respective look-up table for the neural network layer.

公开(公告)号：US20190356931A1

公开(公告)日：2019-11-21

申请号：US16413992

申请日：2019-05-16

Applicant: GOOGLE LLC

Inventor： David Marwood ,  Michele Covell ,  Shumeet Baluja ,  Nicholas Milo Johnston ,  Pascal Massimino

IPC: H04N19/90 ,  H04N19/186 ,  H04N19/176 ,  G06T9/00

Abstract: An encoder system can include a pixel grid generator to receive an image having a first dimension, generate a grid having a second dimension, add a plurality of points to positions on the grid, and map a plurality of pixels of the image to the plurality of points. The encoder system can include a color module to assign a color to each of the plurality of points using a color table, a triangulation module to generate a plurality of vertices based on the plurality of points and triangulate the grid using the vertices, and a compression module to compress the vertices as a set of compressed vertex positions and a set of vertex colors.

公开(公告)号：US20190238893A1

公开(公告)日：2019-08-01

申请号：US15883639

申请日：2018-01-30

Applicant: GOOGLE LLC

Inventor： Michele Covell ,  David Marwood ,  Shumeet Baluja ,  Rahul Sukthankar

IPC: H04N19/91 ,  H04N19/44

CPC classification number: H04N19/91 ,  H04N19/13 ,  H04N19/14 ,  H04N19/176 ,  H04N19/44 ,  H04N19/463

Abstract: Encoding and decoding occupancy information is disclosed. A method includes determining row sums for the region, determining column sums for the region, encoding, in a compressed bitstream, at least one of the row sums and the column sums, and encoding, in the compressed bitstream and based on a coding order, at least one of the rows and the columns of the region. The coding order is based on the encoded at least one of the row sums and the column sums. The row sums include, for each row of the region, a respective count of a number of locations in the row having a specified value. The column sums include, for each column of the region, a respective count of a number of locations in the column having the specified value. A location having the specified value is indicative of the occupancy information at the location.

公开(公告)号：US12118466B2

公开(公告)日：2024-10-15

申请号：US17978026

申请日：2022-10-31

Applicant: Google LLC

Inventor： Michele Covell ,  David Marwood ,  Shumeet Baluja ,  Nicholas Johnston

IPC: G06F1/03 ,  G06N3/048 ,  G06N3/08

CPC classification number: G06N3/08 ,  G06F1/03 ,  G06N3/048

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing a network input using a neural network to generate a network output for the network input. One of the methods includes maintaining, for each of the plurality of neural network layers, a respective look-up table that maps each possible combination of a quantized input index and a quantized weight index to a multiplication result; and generating a network output from a network input, comprising, for each of the neural network layers: receiving data specifying a quantized input to the neural network layer, the quantized input comprising a plurality of quantized input values; and generating a layer output for the neural network layer from the quantized input to the neural network layer using the respective look-up table for the neural network layer.

公开(公告)号：US20230186082A1

公开(公告)日：2023-06-15

申请号：US17978026

申请日：2022-10-31

Applicant: Google LLC

Inventor： Michele Covell ,  David Marwood ,  Shumeet Baluja ,  Nicholas Johnston

IPC: G06N3/08 ,  G06F1/03 ,  G06N3/048

CPC classification number: G06N3/08 ,  G06F1/03 ,  G06N3/048

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing a network input using a neural network to generate a network output for the network input. One of the methods includes maintaining, for each of the plurality of neural network layers, a respective look-up table that maps each possible combination of a quantized input index and a quantized weight index to a multiplication result; and generating a network output from a network input, comprising, for each of the neural network layers: receiving data specifying a quantized input to the neural network layer, the quantized input comprising a plurality of quantized input values; and generating a layer output for the neural network layer from the quantized input to the neural network layer using the respective look-up table for the neural network layer.

公开(公告)号：US11019366B2

公开(公告)日：2021-05-25

申请号：US16413992

申请日：2019-05-16

Applicant: GOOGLE LLC

Inventor： David Marwood ,  Michele Covell ,  Shumeet Baluja ,  Nicholas Milo Johnston ,  Pascal Massimino

IPC: G06T9/00 ,  H04N19/90 ,  H04N19/186 ,  H04N19/176

Abstract: An encoder system can include a pixel grid generator to receive an image having a first dimension, generate a grid having a second dimension, add a plurality of points to positions on the grid, and map a plurality of pixels of the image to the plurality of points. The encoder system can include a color module to assign a color to each of the plurality of points using a color table, a triangulation module to generate a plurality of vertices based on the plurality of points and triangulate the grid using the vertices, and a compression module to compress the vertices as a set of compressed vertex positions and a set of vertex colors.

公开(公告)号：US10681388B2

公开(公告)日：2020-06-09

申请号：US15883639

申请日：2018-01-30

Applicant: GOOGLE LLC

Inventor： Michele Covell ,  David Marwood ,  Shumeet Baluja ,  Rahul Sukthankar

IPC: H04N19/44 ,  H04N19/463 ,  H04N19/91 ,  H04N19/176 ,  H04N19/14 ,  H04N19/13

Abstract: Encoding and decoding occupancy information is disclosed. A method includes determining row sums for the region, determining column sums for the region, encoding, in a compressed bitstream, at least one of the row sums and the column sums, and encoding, in the compressed bitstream and based on a coding order, at least one of the rows and the columns of the region. The coding order is based on the encoded at least one of the row sums and the column sums. The row sums include, for each row of the region, a respective count of a number of locations in the row having a specified value. The column sums include, for each column of the region, a respective count of a number of locations in the column having the specified value. A location having the specified value is indicative of the occupancy information at the location.