Disclosed are a method and apparatus for encoding and decoding an image. The image encoding method includes: generating a first frequency coefficient matrix by performing spatial transform (e.g. discrete cosine transform (DCT) on a current block; determining an angle parameter based on whether the current block is intra or inter predicted; generating a second frequency coefficient matrix by performing a partial switch between at least one of rows and columns of the first frequency coefficient matrix based on the determined angle parameter; and encoding the second frequency coefficient matrix and information about the angle parameter.