A cell lineage can be constructed in a less labor intensive manner and in less time by using a computer. Areas, which are misidentified as nuclei, are efficiently removed. The invention comprises the steps of: selecting one or more nuclei for making a forward link from extracted nucleus candidates, in a plurality of 4D microscopic images in which the nucleus candidates are extracted; obtaining forward linkage information by sequentially extracting a nucleus candidate that meets a predetermined link condition with regard to the selected one or more nuclei from said 4D images; selecting one or more nuclei for making a reverse link from the forward linkage information; and obtaining backward linkage information by sequentially extracting a nucleus candidate that meets a predetermined link condition with regard to the selected one or more nuclei for backward link from said 4D images, removing misidentified nucleus candidates from the forward link information based on the reverse link information. The cell lineage is constructed simultaneously at the time of obtaining the forward link information and reverse link information.