The invention relates to a method for acquiring an element image based on stereo matching in a panoramic imaging system and belongs to the field of stereo display. The method comprises the following steps: using a CCD (charge coupled device) camera array as a mode for the panoramic imaging system to acquire a space image, and converting a plurality of acquired different viewpoint images into a 2D(two-dimensional) element image set used for the displaying of the panoramic imaging system based on the stereo matching principle, wherein the image set has a mode as same as the mode of the image acquired by using the traditional acquiring device, namely a micro-lens array, in the panoramic imaging system, thereby being compatible with the common display device of the panoramic imaging system and completely eliminating the cross interference which occurs when the micro-lens array in the panoramic imaging system is used for acquiring three-dimensional scene images.