Predicting acceleration in a hybrid system may comprise determining a number of cross system calls in a first host-accelerator computer architecture running a workload. Host machine overhead and accelerator overhead in the first host-accelerator computer architecture associated with each of the cross system calls may be determined. Communication delay associated with each of the cross system calls in the first host-accelerator computer architecture running a workload may be determined. An application response time may be predicted for a candidate application to be run in a second host-accelerator computer architecture, based at least on the determined host machine overhead, the accelerator overhead, and the communication delay associated with each of the cross system calls in the first host-accelerator computer architecture running a workload.