POSITION-BIAS CORRECTION FOR PREDICTIVE AND RANKING SYSTEMS

    公开(公告)号:US20240152955A1

    公开(公告)日:2024-05-09

    申请号:US17976089

    申请日:2022-10-28

    CPC classification number: G06Q30/0243 G06F16/9577 G06Q30/0277

    Abstract: Methods, systems, and computer programs are presented for eliminating bias while training an ML model using training data that includes past experimental data. One method includes accessing experiment results, for A/B testing of a first model, that comprise information regarding engagement with a first set of items presented to users, each item being presented within an ordered list of results. A position bias is calculated for positions within the ordered list of results where the items were presented. A machine-learning program is trained to obtain a second model using a training set comprising values for features that include the calculated position bias. The method includes detecting a second set of items to be ranked for presentation to a first user, and calculates, using the second model, a relevance score for the second set of items, which are ranked based on the respective relevance score and presented on a display.

    Delayed processing for over-delivery determination for content delivery system experimentation

    公开(公告)号:US11238488B2

    公开(公告)日:2022-02-01

    申请号:US16824643

    申请日:2020-03-19

    Abstract: A delayed grouping (batch) processing of previous campaign delivery pacing decisions and corresponding outcomes (deliveries) is used to configure a new auction experiment iteration. In the new iteration, a campaign that was previously over-delivered is classified as either (a) over-delivered due to incorrect pacing or (b) over-delivered due to auction experiment design. After the delayed processing, the new auction experiment iteration is conducted with a mitigating action taken on the previously over-delivered campaign if the campaign is classified as (b) over-delivered due to auction experiment design. For example, the mitigating action can include removing the campaign from a subsequent iteration of the experiment, or the experiment can be redesigned. By doing so, the over-delivery caused by the campaign due to the auction experiment design is avoided when performing the new auction experiment iteration.

    DELAYED PROCESSING FOR OVER-DELIVERY DETERMINATION FOR CONTENT DELIVERY SYSTEM EXPERIMENTATION

    公开(公告)号:US20210295374A1

    公开(公告)日:2021-09-23

    申请号:US16824643

    申请日:2020-03-19

    Abstract: A delayed grouping (batch) processing of previous campaign delivery pacing decisions and corresponding outcomes (deliveries) is used to configure a new auction experiment iteration. In the new iteration, a campaign that was previously over-delivered is classified as either (a) over-delivered due to incorrect pacing or (b) over-delivered due to auction experiment design. After the delayed processing, the new auction experiment iteration is conducted with a mitigating action taken on the previously over-delivered campaign if the campaign is classified as (b) over-delivered due to auction experiment design. For example, the mitigating action can include removing the campaign from a subsequent iteration of the experiment, or the experiment can be redesigned. By doing so, the over-delivery caused by the campaign due to the auction experiment design is avoided when performing the new auction experiment iteration.

    Position-bias correction for predictive and ranking systems

    公开(公告)号:US12265987B2

    公开(公告)日:2025-04-01

    申请号:US17976089

    申请日:2022-10-28

    Abstract: Methods, systems, and computer programs are presented for eliminating bias while training an ML model using training data that includes past experimental data. One method includes accessing experiment results, for A/B testing of a first model, that comprise information regarding engagement with a first set of items presented to users, each item being presented within an ordered list of results. A position bias is calculated for positions within the ordered list of results where the items were presented. A machine-learning program is trained to obtain a second model using a training set comprising values for features that include the calculated position bias. The method includes detecting a second set of items to be ranked for presentation to a first user, and calculates, using the second model, a relevance score for the second set of items, which are ranked based on the respective relevance score and presented on a display.

Patent Agency Ranking