A system for detecting malware includes a processor to collect processor trace information corresponding to an application being executed by the processor (202). The processor can also detect an invalid indirect branch instruction from the processor trace information (204) and detect at least one malware instruction being executed by the application in response to analyzing modified memory values corresponding to the invalid indirect branch (206). Additionally, the processor can block the application from accessing or modifying memory (208).