公开(公告)号：US20230153700A1

公开(公告)日：2023-05-18

申请号：US17983130

申请日：2022-11-08

Applicant: Google LLC

Inventor： Erik Michael Lindgren ,  Sashank Jakkam Reddi ,  Ruiqi Guo ,  Sanjiv Kumar

IPC: G06N20/20 ,  G06F12/0875 ,  G06F12/0891

CPC classification number: G06N20/20 ,  G06F12/0875 ,  G06F12/0891

Abstract: Provided are systems and methods which more efficiency train embedding models through the use of a cache of item embeddings for candidate items over a number of training iterations. The cached item embeddings can be “stale” embeddings that were generated by a previous version of the model at a previous training iteration. Specifically, at each iteration, the (potentially stale) item embeddings included in the cache can be used when generating similarity scores that are the basis for sampling a number of items to use as negatives in the current training iteration. For example, a Gumbel-Max sampling approach can be used to sample negative items that will enable an approximation of a true gradient. New embeddings can be generated for the sampled negative items and can be used to train the model at the current iteration.

公开(公告)号：US11354287B2

公开(公告)日：2022-06-07

申请号：US16715620

申请日：2019-12-16

Applicant: GOOGLE LLC

Inventor： Xiang Wu ,  David Morris Simcha ,  Sanjiv Kumar ,  Ruiqi Guo

IPC: G06F16/20 ,  G06F16/22 ,  G06F16/27 ,  G06F16/248 ,  G06F16/28 ,  G06F17/16 ,  G06F16/2458

Abstract: Techniques of indexing a database and processing a query involve decomposing the residual term according to a projection matrix that is based on a given direction v. For example, for each database element of a partition, the residual for that database element is split into a component parallel to a given direction and a component perpendicular to that direction. The parallel component lies in a one-dimensional subspace spanned by the direction and may be efficiently quantized with a scalar quantization. The perpendicular component is quantized using multiscale quantization techniques. The quantized residual components and the center elements of each partition define the indexed database. Upon receipt of a query from a user, the inner products of q with the residual may be computed efficiently using the quantized residual components. From these inner products, the database elements that are most similar to the query are selected and returned to the user.

公开(公告)号：US20230123941A1

公开(公告)日：2023-04-20

申请号：US18081376

申请日：2022-12-14

Applicant: Google LLC

Inventor： Xiang Wu ,  David Simcha ,  Daniel Holtmann-Rice ,  Sanjiv Kumar ,  Ananda Theertha Suresh ,  Ruiqi Guo ,  Xinnan Yu

IPC: G06F16/33 ,  G06F16/31 ,  G06N20/00

Abstract: The present disclosure provides systems and methods that include or otherwise leverage use of a multiscale quantization model that is configured to provide a quantized dataset. In particular, the multiscale quantization model can receive and perform vector quantization of a first dataset. The multiscale quantization model can generate a residual dataset based at least in part on a result of the vector quantization. The multiscale quantization model can apply a rotation matrix to the residual dataset to generate a rotated residual dataset that includes a plurality of rotated residuals. The multiscale quantization model can perform reparameterization of each rotated residual in the rotated residual dataset into a direction component and a scale component. The multiscale quantization model can perform product quantization of the direction components of the plurality of rotated residuals, and perform scalar quantization of the scale components of the plurality of rotated residuals.

公开(公告)号：US20210064634A1

公开(公告)日：2021-03-04

申请号：US17001850

申请日：2020-08-25

Applicant: Google LLC

Inventor： Ruiqi Guo ,  David Simcha ,  Quan Geng ,  Felix Chern ,  Sanjiv Kumar ,  Xiang Wu

IPC: G06F16/25 ,  G06F16/2457 ,  H03M7/30

Abstract: Generally, the present disclosure is directed to systems and methods of quantizing a database with respect to a novel loss or quantization error function which applies a weight to an error measurement of quantized elements respectively corresponding to the datapoints in the database. The weight is determined based on the magnitude of an inner product between the respective datapoints and a query compared therewith. In contrast to previous work, embodiments of the proposed loss function are responsive to the expected magnitude of an inner product between the respective datapoints and a query compared therewith and can prioritize error reduction for higher-ranked pairings of the query and the datapoints. Thus, the systems and methods of the present disclosure provide solutions to some of the problems with traditional quantization approaches, which regard all error as equally impactful.

公开(公告)号：US10394777B2

公开(公告)日：2019-08-27

申请号：US14951909

申请日：2015-11-25

Applicant: Google LLC

Inventor： Xinnan Yu ,  Sanjiv Kumar ,  Ruiqi Guo

IPC: G06F17/00 ,  G06F16/22 ,  G06F16/33 ,  G06F16/951

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for efficiently performing linear projections. In one aspect, a method includes actions for obtaining a plurality of content items from one or more content sources. Additional actions include, extracting a plurality of features from each of the plurality of content items, generating a feature vector for each of the extracted features in order to create a search space, generating a series of element matrices based upon the generated feature vectors, transforming the series of element matrices into a structured matrix such that the transformation preserves one or more relationships associated with each element matrix of the series of element matrices, receiving a search object, searching the enhanced search space based on the received search object, provided one or more links to a content item that are responsive to the search object.

公开(公告)号：US20190114343A1

公开(公告)日：2019-04-18

申请号：US15783685

申请日：2017-10-13

Applicant: Google LLC

Inventor： Ruiqi Guo ,  Bo Dai ,  Sanjiv Kumar

IPC: G06F17/30 ,  G06F15/18

Abstract: The present disclosure provides systems and methods that perform stochastic generative hashing. According to one example aspect, a machine-learned hashing model that generates a binary hash for an input can be trained in conjunction with a machine-learned generative model that reconstructs the input from the binary hash. The present disclosure provides a novel generative approach to learn hash functions through Minimum Description Length principle such that the learned hash codes maximally compress the dataset. According to another example aspect, the present disclosure provides an efficient learning algorithm based on the stochastic distributional gradient, which avoids the notorious difficulty caused by binary output constraints, to jointly optimize the parameters of the hashing model and the associated generative model. The present disclosure also provides extensive experiments which show that the systems and methods described herein achieve better retrieval results than the existing state-of-the-art methods.