A method for segmenting an image using a CNN is provided. The method includes steps of: a segmentation device acquiring (i) a first segmented image for a t -th frame by a CNN_PREVIOUS, having at least one first weight learned at a t -( i +1)-th frame, segmenting the image, (ii) optical flow images corresponding to the ( t -1)-th to the ( ti )-th frames, including information on optical flows from pixels of the first segmented image to corresponding pixels of segmented images of the ( t -1)-th to the ( t-i ) - th frames, and (iii) warped images for the t -th frame by replacing pixels in the first segmented image with pixels in the segmented images referring to the optical flow images, (iv) losses by comparing the first segmented image with the warped images, (v) a CNN_CURRENT with at least one second weight obtained by adjusting the first weight to thereby segment an image of the t -th frame and frames thereafter by using the CNN_CURRENT.