Detection of an encryption or compression application program may be based on similarity between read files read by a process of the application program and write files written by the process. Read fingerprints of the read files and write fingerprints of the write files are generated. A listing of the read fingerprints is searched for presence of matching write fingerprints to find matched fingerprints. The similarity is calculated based on the read fingerprints and matched fingerprints.