In some embodiments, a method analyzes content of a video to generate a representation for a video that represents a relationship of a bitrate to a quality measurement for the video. A plurality of candidate points on the representation are selected. Then, the method selects a profile ladder from at least a portion of the plurality of candidate points. Each profile in the profile ladder is associated with a candidate point and the profile includes a corresponding bitrate and a quality measurement for the respective candidate point. The profile ladder is used for the video.