Video overlays are modified to enhance readability. In one example a frame of a video sequence of frames is received and an overlay for display on the frame is received. A location of the overlay in the frame is determined. Background pixels in the frame that are in the overlay location are determined. Values of the identified pixels are analyzed and an overlay color is selected based on the analysis. The overlay is then blended into the frame using the selected overlay color.