An apparatus and method for encoding video using directional discrete waveform transforms (DDWT), such as within a codec device. DDWT can be utilized to replace the use of intra transforms and inter transforms within the encoding system. In many ways the output of the DDWT can be compared with that provided using MDDT, however, it does not require a training process while it also provides enhanced encoding of feature edges with desirable visual characteristics. The transforms are applied in at least two passes, along the prediction direction, and then across the prediction direction, instead of being applied in fixed vertical and horizontal directions. Directional scaling is not required prior to the second stage of transforms.