The present invention relates to a method and apparatus for encoding or decoding a video signal. The method for processing a video signal according to the present invention determines the type of a block in accordance with the information which indicates whether to partition or merge the current block, acquires predicted values of each sub-block using restored peripheral pixels for each sub-block, and restores the current block using the acquired values. Here, pixels used for acquiring predicted values may include restored pixels for other sub-blocks contained in the current block, and the type of sub-blocks can be taken into account in a prediction mode for acquiring predicted values.