Some embodiments relate to sharpening segments of an image differently based on content in the image. Content based sharpening is performed by a content image processing circuit that receives luminance values of an image and a content map. The content map identifies categories of content in segments of the image. Based on one or more of the identified categories of content, the circuit determines a content factor associated with a pixel. The content factor may also be based on a texture and/or chroma values. A texture value indicates a likelihood of a category of content and is based on detected edges in the image. A chroma value indicates a likelihood of a category of content and is based on color information of the image. The circuit receives the content factor and applies it to a version of the luminance value of the pixel to generate a sharpened version of the luminance value.