Systems and methods for removing or suppressing information in images and video frames is described herein. In particular, systems and methods provide for removing information from images capable of identifying individuals related to the image. For example, embodiments provide for the removal of protected health information (PHI) from source images, including medical images, video frames, and documents converted to images or video frames. In addition, embodiments operate on actual images and video frames as opposed to data extracted from such sources. In particular, embodiments provide for the creation of a PHI filter for an individual of interest comprised of identifying information. Images are filtered using the PHI filter and information potentially identifying the individual of interest is located and removed from the image.