Hybrid inspectors are provided. One system includes computer subsystem(s) configured for receiving optical based output and electron beam based output generated for a specimen. The computer subsystem(s) include one or more virtual systems configured for performing one or more functions using at least some of the optical based output and the electron beam based output generated for the specimen. The system also includes one or more components executed by the computer subsystem(s), which include one or more models configured for performing one or more simulations for the specimen. The computer subsystem(s) are configured for detecting defects on the specimen based on at least two of the optical based output, the electron beam based output, results of the one or more functions, and results of the one or more simulations.