Entity scoring calibration includes receiving values for a metric by which to calibrate a reputation scoring model. The reputation scoring model is usable to determine a reputation score. The reputation scoring model to be calibrated is based at least in part on a set of model parameters associated with reputation scoring components. It further includes receiving a plurality of feedback items pertaining to one or more entities. It further includes calibrating the reputation scoring model at least in part by adjusting at least some of the model parameters associated with the reputation scoring components such that reputation scores generated by the calibrated reputation scoring model track the values for the metric.