A video encoder (70) for coding moving pictures comprising a buffer (16c) with a plurality of memory areas capable of storing frames composed of top fields and bottom fields, a motion estimation unit (19) operable to code, field by field, inputted pictures performing moving estimation and moving compensation by referring, field by field, to the picture data stored in a memory area, a motion compensation unit (16d), a subtractor (11), a transformation unit (13) and a quantization unit (14), a memory management unit (71) operable to manage, frame by frame, a plurality of memory areas, an inverse quantization unit (16a) and inverse discrete cosine transform unit (16b) operable to decode picture data in coded fields and store the picture data in the decoded field in any of the plurality of memory areas under the management by the memory management unit (71).