A layout analysis program, a layout analysis apparatus, layout analysis method and a medium can highly accurately extract a text block from an image if the image is a color image. The layout analysis program causes a computer to execute a divided region extracting step that extracts a region partitioned by a pattern according to a binary image so as to use the outcome of extraction as divided region, a set of character elements extracting step that extracts a set of the character elements extracted by a first binary image layout analysis process for each extracted divided region so as to use the outcome of extraction as set of character elements, a text block extracting step that extracts a region including the extracted set of character elements in each divided region so as to avoid overlapping the non-character elements extracted by a second binary image layout analysis process and use the outcome of extraction as text block and a layout information generating step that generates layout information according to the text block and the non-character elements extracted by the second binary image layout analysis process.