发明申请
- 专利标题： Optimized Judge Assignment under Constraints
- 专利标题（中）： 在限制下优化判决分配
-
申请号： US13314676申请日： 2011-12-08
-
公开(公告)号： US20130152091A1公开(公告)日： 2013-06-13
- 发明人: Chao Liu , Frederick Bruce Shepherd
- 申请人： Chao Liu , Frederick Bruce Shepherd
- 申请人地址： US WA Redmond
- 专利权人： MICROSOFT CORPORATION
- 当前专利权人： MICROSOFT CORPORATION
- 当前专利权人地址： US WA Redmond
- 主分类号： G06F9/46
- IPC分类号： G06F9/46
摘要：
Described is a technology by which an assignment model is computed to distribute labeling tasks among judging entities (judges). The assignment model is optimized by obtaining accuracy-related data of the judges, e.g., by probing the judges with labeling tasks having a gold standard label and evaluating the judges' labels against the gold standard labels, and optimizing for accuracy. Optimization may be based upon on or more other constraints, such as per-judge cost and/or quota.