A parallel processor comprises a plurality of proces­sing units (13) connected to each other via input/output ports (132). Each processing unit comprises: a local memory (131) for storing programs and data; a local bus for inputting/outputting the program and data to and from the memory and consisting of an address signal line, a data signal line, and a control signal line; a CPU (130) for reading the program from the local memory via the local bus as to process the program, for reading data needed to perform the processing from the memory via the local bus, and for making data which has been updated due to the processing to be stored again by the local memory through the local bus; a plurality of input/output ports (132) capable of connecting the local bus to a plurality of outside buses and needed for the CPU to input/output data to and from another memory of another processing unit or for another CPU of another processing unit to input/output data to and from this local memory; and bus switches for connec­ting two of the outside buses for the purpose of enabling data transfers between the CPU and the memory which are dis­posed on individual buses to be performed, wherein a main CPU (10) for setting program and data in the plural proces­sing units and for recovering data and the plural processing units are connected to the main CPU through at least one input/output port of the processing units.