公开(公告)号：US20230153700A1

公开(公告)日：2023-05-18

申请号：US17983130

申请日：2022-11-08

Applicant: Google LLC

Inventor： Erik Michael Lindgren ,  Sashank Jakkam Reddi ,  Ruiqi Guo ,  Sanjiv Kumar

IPC: G06N20/20 ,  G06F12/0875 ,  G06F12/0891

CPC classification number: G06N20/20 ,  G06F12/0875 ,  G06F12/0891

Abstract: Provided are systems and methods which more efficiency train embedding models through the use of a cache of item embeddings for candidate items over a number of training iterations. The cached item embeddings can be “stale” embeddings that were generated by a previous version of the model at a previous training iteration. Specifically, at each iteration, the (potentially stale) item embeddings included in the cache can be used when generating similarity scores that are the basis for sampling a number of items to use as negatives in the current training iteration. For example, a Gumbel-Max sampling approach can be used to sample negative items that will enable an approximation of a true gradient. New embeddings can be generated for the sampled negative items and can be used to train the model at the current iteration.