A PCM audio signal recording and reproducing apparatus samples an audio signal accompanying with a video signal, converts it to a PCM signal, adds an error detection correction data to a predetermined number of PCM data, forms a data frame by those signals, modulates it in accordance with a predetermined modulation system, and records the audio signal together with the video signal or singly on a record medium by using a rotary head type scanner. A digital signal processing circuit having a memory of a predetermined capacity is provided, and the number of digital data to be recorded in one data frame is varied in accordance with a difference between the number of input data of the digital audio signal to the digital signal processing circuit and the number of output data supplied to the record medium for recording. In the recording mode, the PCM audio signal is sectioned into blocks, a data block address signal is added to each block to indicate a relative order to other blocks and complete in one track, certain number of blocks are combined to form a block set for processing signals, and an identification signal completing in the block and an interleave block address signal for indicating a relative order to other blocks are added to each block.