The invention relates to a structure feature based on Chinese painting and calligraphy seal image automatic extracting method, comprising the steps as follows: 1) chromatography of an input image; 2) image segmentation based on the connected region; 3) image region filtration based on the seal structure feature; and 4) seal image extraction in the input image. In the method, Chinese painting and calligraphy chromatography analysis model based on machine learning is adopted to realize the image segmentation of painting and calligraphy works based on granularity level of the homogeneous region of the color layers. On this basis, a painting and calligraphy seal image extracting method based on seal outline template and internal structure complexity is provided, thereby realizing a content-based Chinese painting and calligraphy image retrieval and semantic annotation system with the seal image as the key information so as to lay a foundation for the retrieval, appraisement, classificationand the like of the Chinese painting and calligraphy works.