According to an embodiment, a compiling device compiling a source program written so as to use a frame memory includes a processing delay amount calculator configured to calculate respective processing delay amounts between a plurality of process tasks in the source program on the basis of processing states of pieces of data processed by the process tasks. The compiling device also includes a line memory amount calculator configured to calculate respective line memory sizes required for each of the process tasks on the basis of an access range of a frame memory from which the process task reads data and an instruction code converter configured to convert the plurality of process tasks to instruction codes executable in a pipeline on the basis of the processing delay amounts and the line memory sizes.