In a semiconductor device performing pipeline processing with the use of a reading portion reading an instruction and an arithmetic portion performing an operation in accordance with the instruction, the instruction held in the reading portion is transmitted from the flip-flop to the memory when branch prediction turns out to be wrong. Note that the arithmetic portion controls transmission and reception of the instruction between the flip-flop and the memory which are included in the reading portion. This enables elimination of redundant operations in the reading portion in the case where an instruction read by the reading portion after the branch prediction turns out to be wrong is a subroutine, or the like. That is, the instruction held in the memory is transmitted back to the flip-flop without rereading of the same instruction by the reading portion, whereby the instruction can be output to the arithmetic portion.