Systems and methods for branch prediction include identifying a subset of branch instructions from an execution trace of instructions executed by a processor. The identified subset of branch instructions have greater benefit from branch predictions made by a neural branch predictor than branch predictions made by a non-neural branch predictor. During runtime, the neural branch predictor is selectively used for obtaining branch predictions of the identified subset of branch instructions. For remaining branch instructions outside the identified subset of branch instructions, branch predictions are obtained from a non-neural branch predictor. Further, a weight vector matrix comprising weight vectors for the identified subset of branch instructions of the neural branch predictor is pre-trained based on the execution trace.