A display may have an array of display pixels to display images. Digital display data may be received by a digital-to-analog converter. The digital-to-analog converter can convert the digital display data to analog display data for the display pixels. The magnitudes of the analog display data signals that the digital-to-analog converter provides to the display pixels can be controlled by a control signal such as a reference voltage received by the digital-to-analog converter. A brightness controller may have multiple peak luminance control profiles. A brightness setting may be processed by a look-up table to produce information identifying a selected one of the peak luminance control profiles. The brightness controller may use the selected peak luminance control profile and average frame luminance for the digital display data to produce the reference voltage that controls the digital-to-analog controller.