A data processing apparatus executes call instructions, and after a sequence of instructions executed in response to a call instruction a return instruction causes the program flow to return to a point in the program sequence associated with that call instruction. The data processing apparatus is configured to speculatively execute instructions in dependence on a predicted outcome of earlier instructions and a return address prediction unit is configured to store return addresses associated with unresolved call instructions. The return address prediction unit comprises: a stack portion onto which return addresses associated with unresolved call instructions are pushed, and from which a return address is popped when a return instruction is speculatively executed; and a buffer portion which stores an entry for each unresolved call instruction executed and for each return instruction which is speculatively executed.