A method of encoding a video signal using a graph-based transformation includes: generating a residual block using a prediction block generated according to an intra prediction mode; obtaining at least one of a self-loop weight indicating a weight of boundary pixels in the residual block or a correlation coefficient indicating an inter-pixel correlation, on the basis of a prediction angle corresponding to the intra prediction mode; generating a graph on the basis of at least one of the self-loop weight or the correlation coefficient; determining a graph-based transformation kernel on the basis of the graph; and performing a transform for the residual block using the graph-based transformation kernel.