Audio data at the top of each packet is always set at the top of sample data and packets can be treated as units, thus facilitating a timing process, sequential processing and the like for processing audio data. A plurality of samples consisting of variable length data are arranged in a packet of predetermined bytes. In this case, other samples than the first sample in the plurality of samples are sequentially arranged with the top of the first sample being positioned at a predetermined position of the packet, the total byte length of the sample data is equal to or smaller than a maximum byte length of the packet, and when a byte length of the packet is less than the maximum byte length, invalid data of a stuffing byte or a padding byte is inserted in a remaining portion.