Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for ranking content items. In one aspect, a method includes identifying, for a content item, a bid value specifying an amount a content item provider is willing to pay for user interaction with the content item. A predicted performance measure is identified for the content item. The predicted performance measure is adjusted based on a weighting factor for the content item. The weighting factor for the content item is indicative of confidence that the predicted performance measure will match an actual performance measure for the content item and can be different than a weighting factor for another content item identified for inclusion in a ranking with the content item. A rank score is determined for the content item using the bid value and adjusted predicted performance measure. The content item is provided based on the rank score.