An online system selecting content items for presentation to its users accounts for likelihoods of users performing actions associated with content items when selecting content items. The online system maintains models determining likelihoods of users performing various actions. If a content item is associated with an action that infrequently occurs, information for determining the model for the action is limited, so the online system increases a bid amount associated with the content item during a time interval to an amount based on a likelihood of the user performing a more frequently occurring alternative action and an average bid amount for the alternative action from content items previously presented to users. The online system also determines an amount based on the model for the action and the bid amount for during the time interval and stops increasing the bid amount when the rate of change has less than a threshold magnitude.