Provided is a medical image analyzing system and a method thereof, which includes: acquiring a processed image having a segmentation label corresponding to a cancerous part of an organ (if present), generating a plurality of image patches therefrom, performing feature analysis on the image patches and model training to obtain prediction values, drawing a receiver operating characteristic curve using the prediction values, and determining a threshold with which determines whether the image patches are cancerous, so as to effectively improve the detection rate of, for example, pancreatic cancer.