公开(公告)号：EP3152711A1

公开(公告)日：2017-04-12

申请号：EP15729047.9

申请日：2015-06-05

申请人： Microsoft Technology Licensing, LLC

发明人： EDEN, Semiha Ece Kamar ,  PATEL, Rajesh M. ,  SHELFORD, Steven J.R. ,  WU, Hai ,  MOLNAR, David A. ,  HORVITZ, Eric J.

IPC分类号： G06Q10/06 ,  G06Q10/10

CPC分类号： G06Q10/063118 ,  G06Q10/1053

摘要： A crowdsourcing environment is described herein which uses a single-stage or multi-stage approach to evaluate the quality of work performed by a worker, with respect to an identified task. In the multi-stage case, an evaluation system, in the first stage, determines whether the worker corresponds to a spam agent. In a second stage, for a non-spam worker, the evaluation system determines the propensity of the worker to perform desirable (e.g., accurate) work in the future. The evaluation system operates based on a set of features, including worker-focused features (which describe work performed by the particular worker), task-focused features (which describe tasks performed in the crowdsourcing environment), and system-focused features (which describe aspects of the configuration of the crowdsourcing environment). According to one illustrative aspect, the evaluation system performs its analysis using at least one model, produced using any type of supervised machine learning technique.

摘要翻译： 本文描述了众包源环境，其使用单阶段或多阶段方法来评估工作人员对所识别的任务执行的工作质量。 在多阶段的情况下，评估系统在第一阶段确定工作人员是否对应于垃圾邮件代理。 在第二阶段，对于非垃圾邮件工作者，评估系统确定工人未来执行理想（例如准确）工作的倾向。 评估系统基于一组功能进行操作，包括以工作人员为重点的功能（描述特定工作人员执行的工作），以任务为中心的功能（描述在众包环境中执行的任务）和系统关注的功能（描述 各方面的配置环境）。 根据一个说明性方面，评估系统使用至少一个使用任何类型的监督机器学习技术产生的模型进行分析。