公开(公告)号：US20240176993A1

公开(公告)日：2024-05-30

申请号：US18485457

申请日：2023-10-12

Applicant: Google LLC

Inventor： Prateek Jain ,  Inderjit Singh Dhillon ,  Fnu Devvrit ,  Aditya Sinha

IPC: G06N3/0464 ,  G06N3/0895

CPC classification number: G06N3/0464 ,  G06N3/0895

Abstract: A method of training a machine learning model includes receiving training data comprising a graph structure and one or more feature attributes and determining an encoded graph based on applying the machine learning model to the graph structure and the one or more feature attributes. The machine learning model comprises a graph convolutional network layer. The encoded graph comprises one or more nodes and one or more paths connecting the one or more nodes. The method also includes selecting a plurality of positive samples through random walks along the one or more paths of the encoded graph, selecting a plurality of negative samples from the encoded graph by randomly sampling the one or more nodes of the encoded graph, determining a loss value, and updating, based on the loss value, one or more learnable parameter values of the machine learning model.

公开(公告)号：US20240054391A1

公开(公告)日：2024-02-15

申请号：US17928372

申请日：2022-04-05

Applicant: GOOGLE LLC

Inventor： Abhradeep Guha Thakurta ,  Li Zhang ,  Prateek Jain ,  Shuang Song ,  Steffen Rendle ,  Steve Shaw-Tang Chien ,  Walid Krichene ,  Yarong Mu

IPC: G06N20/00 ,  G06F21/62

CPC classification number: G06N20/00 ,  G06F21/6218

Abstract: Computer-implemented systems and methods for training a decentralized model for making a personalized recommendation. In one aspect, the method comprising: obtaining, using user activity data, client-side training data that includes features and training labels; and training, by the client device, a decentralized model in training rounds, wherein training, in each training round comprises: receiving, first data including a current server-side embedding generated by the server-side machine learning model, wherein the first data received from the server does not include any server-side data used in generating the current server-side embedding; generating, using the client-side machine learning model, a client-side embedding based on the client-side training data; updating, using the client-side embedding and the current server-side embedding and based on the training labels, the client-side machine learning model; generating, an updated client-side embedding; and transmitting second data including the updated client-side embedding for subsequent updating of the server-side machine learning model.

公开(公告)号：US20240005131A1

公开(公告)日：2024-01-04

申请号：US18343723

申请日：2023-06-28

Applicant: Google LLC

Inventor： Himanshu Jain ,  Lovish Madaan ,  Prateek Jain ,  Venkata Sesha Pavana Srinadh Bhojanapalli

IPC: G06N3/0455

CPC classification number: G06N3/0455

Abstract: Systems and methods for processing inputs using attention neural networks with tree attention layers. Each tree attention layer includes one or more tree attention sub-layers that are each configured to: process query vectors using a decision tree model for the tree attention sub-layer to determine a respective tree path for each query vector; process key vectors using the decision tree model to determine a respective tree path for each key vector; and generate an attended input sequence comprising a respective attended input at each of the plurality of input positions, comprising: generating, for each particular input position, the respective attended input at the particular input position based on (i) the tree path for the query vector at the particular input position (ii) the respective tree paths for the key vectors at each of the plurality of input positions and (iii) the value vectors at a subset of the input positions.