-
公开(公告)号：US20200372435A1
公开(公告)日：2020-11-26
申请号：US16051105
申请日：2018-07-31
Applicant: Microsoft Technology Licensing, LLC
Inventor： Krishnaram Kenthapadi , Sahin C. Geyik , Stuart M. Ambler
Abstract: The disclosed embodiments provide a system for achieving fairness across multiple attributes in a ranking. During operation, the system obtains a ranking of recommended candidates outputted by a machine learning model in response to a request. Next, the system obtains target proportions of multiple attribute values in the ranking of recommended candidates. The system then generates, based on the ranking, a set of attribute-specific rankings of recommended candidates, wherein each attribute-specific ranking includes candidates with a common attribute value. The system also generates, based on the attribute-specific rankings and one or more ranking criteria associated with the target proportions, a reranking of recommended candidates. Finally, the system outputs at least a portion of the reranking in a response to the request.
-
公开(公告)号：US20200349605A1
公开(公告)日：2020-11-05
申请号：US16401832
申请日：2019-05-02
Applicant: Microsoft Technology Licensing, LLC
Inventor： Sahin C. Geyik , Florian Raudies , Xi Chen , Yu Wang , Wen Pu
Abstract: The disclosed embodiments provide a system for performing calibration of response rates. During operation, the system obtains a position of a content item in a ranking of content items generated for delivery to a member of an online system and a predicted response rate by the member to the content item. Next, the system determines an updated response rate by the member to the content item based on the position of the content item in the ranking and dimensions associated with the predicted response rate and the ranking. The system then outputs the updated response rate for use in managing delivery of the content item.
-
公开(公告)号：US20190188531A1
公开(公告)日：2019-06-20
申请号：US15844833
申请日：2017-12-18
Applicant: Microsoft Technology Licensing, LLC
Inventor： Cagri Ozcaglar , Vijay K. Dialani , Sara S. Gerrard , Sahin C. Geyik , Anish R. Nair
CPC classification number: G06K9/6231 , G06K9/6212 , G06K9/6219 , G06N20/00
Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a set of feature additions and an evaluation metric for assessing the performance of a statistical model. Next, the system automatically builds treatment versions of the statistical model using a set of baseline features for the statistical model and feature combinations generated using the feature additions. The system then uses a hypothesis test and a fixed set of feature values to compare a baseline value of the evaluation metric for a baseline version of the statistical model that is built using the set of baseline features with additional values of the evaluation metric for the treatment versions. Finally, the system outputs a result of the hypothesis test for use in assessing an impact of the feature combinations on a performance of the statistical model.
-
Patent Agency Ranking
公开(公告)号：US20200372472A1
公开(公告)日：2020-11-26
申请号：US16050764
申请日：2018-07-31
Applicant: Microsoft Technology Licensing, LLC
Inventor： Krishnaram Kenthapadi , Sahin C. Geyik , Stuart M. Ambler
Abstract: The disclosed embodiments provide a system for performing multi-level ranking for mitigating machine learning model bias. During operation, the system applies a machine learning model to features for qualified candidates that match parameters of a request to produce a first ranking of recommended candidates. Next, the system calculates a distribution of an attribute in the qualified candidates and generates a first reranking of recommended candidates that more accurately reflects the distribution of the attribute in the qualified candidates. The system then applies another machine learning model to the first reranking to produce a second ranking of recommended candidates and generates a second reranking of recommended candidates that more accurately reflects the distribution of the attribute in the qualified candidates. Finally, the system outputs at least a portion of the second reranking in a response to the request.
-
公开(公告)号：US20200349604A1
公开(公告)日：2020-11-05
申请号：US16401822
申请日：2019-05-02
Applicant: Microsoft Technology Licensing, LLC
Inventor： Sahin C. Geyik , Xi Chen , Yu Wang , Keqing Liang , Wen Pu
Abstract: The disclosed embodiments provide a system that performs pacing for balanced delivery. During operation, the system obtains predicted response rates associated with impressions of a content item delivered within an online system and a cost per action (CPA) for the content item. Next, the system determines an impression-based spending for the content item based on the predicted response rates and the CPA. The system then calculates a pacing score for the content item based on the impression-based spending. Finally, the system adjusts subsequent interactions with the content item based on the pacing score.
-
公开(公告)号：US11068743B2
公开(公告)日：2021-07-20
申请号：US15844833
申请日：2017-12-18
Applicant: Microsoft Technology Licensing, LLC
Inventor： Cagri Ozcaglar , Vijay K. Dialani , Sara S. Gerrard , Sahin C. Geyik , Anish R. Nair
IPC: G06K9/62
Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a set of feature additions and an evaluation metric for assessing the performance of a statistical model. Next, the system automatically builds treatment versions of the statistical model using a set of baseline features for the statistical model and feature combinations generated using the feature additions. The system then uses a hypothesis test and a fixed set of feature values to compare a baseline value of the evaluation metric for a baseline version of the statistical model that is built using the set of baseline features with additional values of the evaluation metric for the treatment versions. Finally, the system outputs a result of the hypothesis test for use in assessing an impact of the feature combinations on a performance of the statistical model.
-
公开(公告)号：US20200372304A1
公开(公告)日：2020-11-26
申请号：US16051115
申请日：2018-07-31
Applicant: Microsoft Technology Licensing, LLC
Inventor： Krishnaram Kenthapadi , Sahin C. Geyik , Stuart M. Ambler
Abstract: The disclosed embodiments provide a system for quantifying machine learning model bias. During operation, the system obtains a set of qualified candidates that match parameters of a request. Next, the system obtains a ranking of recommended candidates outputted by a machine learning model after the qualified candidates are inputted into the machine learning model. The system then generates a first distribution of an attribute in the ranking of recommended candidates and a second distribution of the attribute in the qualified candidates. The system also calculates, based on the first and second distributions, a skew metric representing a difference between a first proportion of the attribute value in the ranking of recommended candidates and a second proportion of the attribute value in the qualified candidates. Finally, the system outputs the skew metric for use in evaluating bias in the machine learning model.
-
公开(公告)号：US20200210908A1
公开(公告)日：2020-07-02
申请号：US16232862
申请日：2018-12-26
Applicant: Microsoft Technology Licensing, LLC
Inventor： Keqing Liang , Wen Pu , Sahin C. Geyik , Yu Wang , Ying Chen , Yin Zhang , Sumedha K. Swamy
Abstract: The disclosed embodiments provide a system for performing dynamic job bidding optimization. During operation, the system obtains historical data containing a time series of interactions with a job. Next, the system uses the historical data to calculate an initial price of a job based on a predicted number of interactions with the job. The system then determines a first dynamic adjustment to the initial price that improves utilization of a budget for the job and a second dynamic adjustment to the initial price that improves a performance of the job. Finally, the system applies the first and second adjustments to the initial price to produce an updated price for the job and delivers the job within an online system based on the updated price.
-
-
-
-
-
-
-
Search Results
8
records
(took 0.012 seconds)
