Methods, systems, and apparatus, including computer programs encoded on computer storage media, for post-hoc deferral for classification tasks. In particular, a system can perform either post-hoc threshold correction or post-hoc rejector training to account for the cost of deferring model inputs to an expert system for classification.