Multi-task neutral network for feed ranking

    公开(公告)号:US10909454B2

    公开(公告)日:2021-02-02

    申请号:US15469550

    申请日:2017-03-26

    Applicant: Facebook, Inc.

    Abstract: For a content item with unknown tasks performed by a viewing user on an online system, the online system receives a plurality of content items associated with a viewing user. The online system derives a feature vector for each content item. The online system predicts a likelihood of interacting with each content item using a prediction model associated with a plurality of tasks. The prediction model comprises a plurality of shared layers and a plurality of separate layers. The plurality of shared layers are configured to extract common features that are shared across the plurality of tasks. Each separate layer is configured to predict likelihood of the viewing user performing a task associated with the separate layer based on the common features. The online system scores each content item based on predicted likelihood of each task. The online system ranks the plurality of content items based on the scoring.

    Mixed machine learning architecture

    公开(公告)号:US11144812B2

    公开(公告)日:2021-10-12

    申请号:US15694707

    申请日:2017-09-01

    Applicant: Facebook, Inc.

    Abstract: A preprocessing module of a neural network has a first input and second input. The module generates multiple, different first latent vector representations of its first input, and multiple, different second latent vector representations of its second input. The module then models pairwise interactions between every unique pairwise combination of the first and second latent vector representations. The module then produces an intermediate output by combining the results of the modeled pairwise interactions.

    Nested machine learning architecture

    公开(公告)号:US11132604B2

    公开(公告)日:2021-09-28

    申请号:US15694695

    申请日:2017-09-01

    Applicant: Facebook, Inc.

    Abstract: In one embodiment, a method includes a preprocessing stage of a neural network model, where the preprocessing stage includes first and second preprocessing modules. Each of the two modules has first input that may receive a dense input and a second input that may receive a sparse input. Each module generates latent vector representations of their respective first and second inputs, and combine the latent vectors with the original first input to define an intermediate output. The intermediate output of the first module is fed into the first input of the second module.

    Nested Machine Learning Architecture
    4.
    发明申请

    公开(公告)号:US20190073586A1

    公开(公告)日:2019-03-07

    申请号:US15694695

    申请日:2017-09-01

    Applicant: Facebook, Inc.

    Abstract: In one embodiment, a method includes a preprocessing stage of a neural network model, where the preprocessing stage includes first and second preprocessing modules. Each of the two modules has first input that may receive a dense input and a second input that may receive a sparse input. Each module generates latent vector representations of their respective first and second inputs, and combine the latent vectors with the original first input to define an intermediate output. The intermediate output of the first module is fed into the first input of the second module.

    Sparse neural network training optimization

    公开(公告)号:US10943171B2

    公开(公告)日:2021-03-09

    申请号:US15694742

    申请日:2017-09-01

    Applicant: Facebook, Inc.

    Abstract: An optimized computer architecture for training an neural network includes a system having multiple GPUs. The neural network may be divided into separate portions, and a different portion is assigned to each of the multiple GPUs. Within each GPU, its portion is further divided across multiple training worker threads in multiple processing cores, and each processing core has lock-free access to a local parameter memory. The local parameter memory of each GPU is separately, and individually, synchronized with a remote master parameter memory by lock memory access. Each GPU has a separate set of communication worker threads dedicated to data transfer between the GPU and the remote parameter memory so that the GPU's training worker threads are not involved with cross GPU communications.

    Mixed Machine Learning Architecture
    6.
    发明申请

    公开(公告)号:US20190073581A1

    公开(公告)日:2019-03-07

    申请号:US15694707

    申请日:2017-09-01

    Applicant: Facebook, Inc.

    Abstract: A preprocessing module of a neural network has a first input and second input. The module generates multiple, different first latent vector representations of its first input, and multiple, different second latent vector representations of its second input. The module then models pairwise interactions between every unique pairwise combination of the first and second latent vector representations. The module then produces an intermediate output by combining the results of the modeled pairwise interactions.

    MULTI-TASK NEUTRAL NETWORK FOR FEED RANKING
    7.
    发明申请

    公开(公告)号:US20180276533A1

    公开(公告)日:2018-09-27

    申请号:US15469550

    申请日:2017-03-26

    Applicant: Facebook, Inc.

    CPC classification number: G06N3/08 G06F16/95

    Abstract: For a content item with unknown tasks performed by a viewing user on an online system, the online system receives a plurality of content items associated with a viewing user. The online system derives a feature vector for each content item. The online system predicts a likelihood of interacting with each content item using a prediction model associated with a plurality of tasks. The prediction model comprises a plurality of shared layers and a plurality of separate layers. The plurality of shared layers are configured to extract common features that are shared across the plurality of tasks. Each separate layer is configured to predict likelihood of the viewing user performing a task associated with the separate layer based on the common features. The online system scores each content item based on predicted likelihood of each task. The online system ranks the plurality of content items based on the scoring.

    Flexible multi-task neutral network for content ranking

    公开(公告)号:US11017287B2

    公开(公告)日:2021-05-25

    申请号:US15784002

    申请日:2017-10-13

    Applicant: Facebook, Inc.

    Inventor: Liang Xiong Yan Zhu

    Abstract: For a content item with unknown tasks performed by a viewing user on an online system, the online system predicts a likelihood of interacting with each content item using a prediction model associated with a plurality of tasks. The prediction model comprises a plurality of independent layers, a plurality of shared layers and a plurality of separate layers. Each independent layer is configured to extract features, for each task, that are not shared across the plurality of tasks. The plurality of shared layers are configured to extract common features that are shared across the plurality of tasks. Each separate layer is configured to predict likelihood of the viewing user performing a task associated with the separate layer based on the features extracted from the plurality of independent layers and the plurality of shared layers.

    Sparse Neural Network Training Optimization
    9.
    发明申请

    公开(公告)号:US20190073590A1

    公开(公告)日:2019-03-07

    申请号:US15694742

    申请日:2017-09-01

    Applicant: Facebook, Inc.

    Abstract: An optimized computer architecture for training an neural network includes a system having multiple GPUs. The neural network may be divided into separate portions, and a different portion is assigned to each of the multiple GPUs. Within each GPU, its portion is further divided across multiple training worker threads in multiple processing cores, and each processing core has lock-free access to a local parameter memory. The local parameter memory of each GPU is separately, and individually, synchronized with a remote master parameter memory by lock memory access. Each GPU has a separate set of communication worker threads dedicated to data transfer between the GPU and the remote parameter memory so that the GPU's training worker threads are not involved with cross GPU communications.

    DEEP SEMANTIC CONTENT SELECTION
    10.
    发明申请

    公开(公告)号:US20180336490A1

    公开(公告)日:2018-11-22

    申请号:US15599240

    申请日:2017-05-18

    Applicant: Facebook, Inc.

    Abstract: To select the content to be presented to the user, a first latent vector is determined for a content item based on a first object associated with the content item. A second latent vector is determined for the content item based on a second object associated with the content item. A content item vector is then determined based on the first and second latent vectors. Furthermore, a user vector is determined based on interactions of the user with the first set of content objects and the second set of content objects. A score indicative of the likelihood of the user interacting with the content item is determined based on the content item vector and the user vector.

Patent Agency Ranking