Disclosed are an intra-prediction mode-based image processing method and apparatus. In particular, the intra-prediction mode-based image processing method may comprise the steps of: generating a first prediction sample by performing spatial prediction on a current chroma block; determining whether or not a combined intra-prediction mode is applicable to the current chroma block; when the combined intra-prediction mode is applicable to the current chroma block, determining a component prediction mode used for inter-component prediction of the current chroma block; generating a second prediction sample by applying the determined component prediction mode to the current chroma block; and generating a final prediction sample of the current chroma block by using the first prediction sample and the second prediction sample.