Provided are an apparatus and a method for encoding a high fidelity video, and an apparatus and a method for decoding a high fidelity video. A video encoding method for encoding first to third planes having different color components includes performing intra prediction in the first plane or inter prediction between each frame for a block of the first plane, and generating a first residue corresponding to a difference between the block of the first plane and the predicted block of the first plane, predicting a block of the second plane using a reconstructed pixel peripheral to the block of the second plane and a reconstructed pixel peripheral to the block of the first plane corresponding to the reconstructed pixel peripheral to the block of the second plane, and generating a second residue corresponding to a difference between the block of the second plane and the predicted block of the second plane, predicting a block of the third plane using a reconstructed pixel peripheral to the block of the third plane and any one of a reconstructed pixel peripheral to the block of the first plane corresponding to the reconstructed pixel peripheral to the block of the third plane and a reconstructed pixel peripheral to the block of the second plane corresponding to the reconstructed pixel peripheral to the block of the third plane, and generating a third residue corresponding to a difference between the block of the third plane and the predicted block of the third plane, and encoding the first to third residues.