An application document known to include malware (such as a document exploit) is opened and executed by its corresponding software application. Behaviors of this document (such as registry, file system, network and process) are monitored and recorded using internal software drivers and hook modules. A behavior report is generated and a baseline pattern is created including a number of regular expressions. A suspicious document of the same type as the monitored document is opened and executed by the same corresponding software application. Behaviors are monitored in the same way and a behavior report is generated. This behavior report is compared to the baseline pattern and a determination is made as to whether a document exploit is present. Known benign documents may also be opened, monitored and their behavior recorded, resulting in creation of a known benign pattern for the corresponding software application.