摘要:
A crowdsourcing environment is described herein which uses a single-stage or multi-stage approach to evaluate the quality of work performed by a worker, with respect to an identified task. In the multi-stage case, an evaluation system, in the first stage, determines whether the worker corresponds to a spam agent. In a second stage, for a non-spam worker, the evaluation system determines the propensity of the worker to perform desirable (e.g., accurate) work in the future. The evaluation system operates based on a set of features, including worker-focused features (which describe work performed by the particular worker), task-focused features (which describe tasks performed in the crowdsourcing environment), and system-focused features (which describe aspects of the configuration of the crowdsourcing environment). According to one illustrative aspect, the evaluation system performs its analysis using at least one model, produced using any type of supervised machine learning technique.
摘要:
Aspects of the subject disclosure are directed towards processing search logs and/or other large scale data sources to detect medical related-effects. For example, an anomalous number of queries regarding a particular symptom and a drug may indicate the existence of a previously unknown side-effect of the drug. Side effects of drug interactions may also be found by processing behavioral data such as queries and social network posts. Also described is the generation of symptom spectra data that is processed to detect anomalies and the like in user behavior corresponding to medical related-effects.