公开(公告)号：US11200284B1

公开(公告)日：2021-12-14

申请号：US15967414

申请日：2018-04-30

Applicant: Facebook, Inc.

Inventor： Miao Li ,  Sagar Chordia ,  Harsh Doshi ,  Xianjie Chen ,  Qin Huang

IPC: G06F16/00 ,  G06F16/904 ,  G06N3/04 ,  G06K9/62 ,  G06F16/901

Abstract: A system trains models to generate embeddings that represent likelihoods associated with features. For example, an embedding may be generated for users and pages such that a user's embedding represents how likely a user is to comment on a given page. Initially, memory space for storing each embedding may be overprovisioned. The system monitors the embeddings for a feature as they are generated and recalculated over time. If the system detects that a particular index value is never updated for embeddings of that feature, then the system may remove that value from the feature embeddings. This allows the array lengths of embeddings to be customized to the particular features they represent, saving memory space. The system may further use related information to identify pooling functions that are most effective for particular features, to identify similarities between entities, and to provide insight into how the feature data influences neural network layers.

公开(公告)号：US20190073581A1

公开(公告)日：2019-03-07

申请号：US15694707

申请日：2017-09-01

Applicant: Facebook, Inc.

Inventor： Xianjie Chen ,  Wenlin Chen ,  Liang Xiong ,  Tianshi Gao

IPC: G06N3/04 ,  G06N3/08

Abstract: A preprocessing module of a neural network has a first input and second input. The module generates multiple, different first latent vector representations of its first input, and multiple, different second latent vector representations of its second input. The module then models pairwise interactions between every unique pairwise combination of the first and second latent vector representations. The module then produces an intermediate output by combining the results of the modeled pairwise interactions.

公开(公告)号：US11144812B2

公开(公告)日：2021-10-12

申请号：US15694707

申请日：2017-09-01

Applicant: Facebook, Inc.

Inventor： Xianjie Chen ,  Wenlin Chen ,  Liang Xiong ,  Tianshi Gao

IPC: G06N3/04 ,  G06N3/08 ,  H04L29/08

Abstract: A preprocessing module of a neural network has a first input and second input. The module generates multiple, different first latent vector representations of its first input, and multiple, different second latent vector representations of its second input. The module then models pairwise interactions between every unique pairwise combination of the first and second latent vector representations. The module then produces an intermediate output by combining the results of the modeled pairwise interactions.

公开(公告)号：US11132604B2

公开(公告)日：2021-09-28

申请号：US15694695

申请日：2017-09-01

Applicant: Facebook, Inc.

Inventor： Xianjie Chen ,  Wenlin Chen ,  Liang Xiong ,  Tianshi Gao

IPC: G06N3/08 ,  G06N7/00

Abstract: In one embodiment, a method includes a preprocessing stage of a neural network model, where the preprocessing stage includes first and second preprocessing modules. Each of the two modules has first input that may receive a dense input and a second input that may receive a sparse input. Each module generates latent vector representations of their respective first and second inputs, and combine the latent vectors with the original first input to define an intermediate output. The intermediate output of the first module is fed into the first input of the second module.

公开(公告)号：US20190073586A1

公开(公告)日：2019-03-07

申请号：US15694695

申请日：2017-09-01

Applicant: Facebook, Inc.

Inventor： Xianjie Chen ,  Wenlin Chen ,  Liang Xiong ,  Tianshi Gao

IPC: G06N3/08 ,  G06N7/00

Abstract: In one embodiment, a method includes a preprocessing stage of a neural network model, where the preprocessing stage includes first and second preprocessing modules. Each of the two modules has first input that may receive a dense input and a second input that may receive a sparse input. Each module generates latent vector representations of their respective first and second inputs, and combine the latent vectors with the original first input to define an intermediate output. The intermediate output of the first module is fed into the first input of the second module.