A system and method for predicting disease outcome by analyzing a large, heterogeneous image by a boosted, multi-field-of-view (FOV) framework, based on image-based features from multi-parametric heterogeneous images, comprises (a) inputting the heterogeneous image; (b) generating a plurality of FOVs at a plurality of fixed FOV sizes, the method for generating the plurality of FOVs at a plurality of fixed FOV sizes comprising dividing simultaneously, via the computing device, the large, heterogeneous image into (i) a plurality of FOVs at a first fixed FOV size from among the plurality of fixed FOV sizes; and (ii) a plurality of FOVs at a second fixed FOV size from among the plurality of fixed FOV sizes; (c) producing simultaneously for the heterogeneous image a combined class decision for: (i) the plurality of FOVs at the first fixed FOV size, and (ii) the plurality of FOV s at the second fixed FOV size.