The embodiments of the present invention provide a method and a device for encoding and decoding videos, and relate to the communication field, and an efficient transformation matrix corresponding to features of each residual block is selected for transformation, which therefore improves encoding efficiency. The solution provided in an embodiment of the present invention is: generating a prediction residual according to input video data; selecting a set of best transformation matrixes among multiple candidate transformation matrixes according to an intra-frame prediction mode and rate-distortion criteria to perform transform-coding on the prediction residual and obtain a transformation result; and generating an encoded stream according to the transformation result and selected transformation matrix index information.