An efficient digital video (DV) decoder process that utilizes a specially constructed quantization matrix allowing an inverse quantization subprocess to perform parallel computations, e.g., using SIMD processing, to efficiently produce a matrix of DCT coefficients. The present invention utilizes a first look-up table (for 8×8 DCT) which produces a 15-valued quantization scale based on class number information and a QNO number for an 8×8 data block (“data matrix”) from an input encoded digital bit stream to be decoded. The 8×8 data block is produced from a deframing and variable length decoding subprocess. An individual 8-valued segment of the 15-value output array is multiplied by an individual 8-valued segment, e.g., “a row,” of the 8×8 data matrix to produce an individual row of the 8×8 matrix of DCT coefficients (“DCT matrix”). The above eight multiplications can be performed in parallel using a SIMD architecture to simultaneously generate a row of eight DCT coefficients. In this way, eight passes through the 8×8 block are used to produce the entire 8×8 DCT matrix, in one embodiment consuming only 33 instructions per 8×8 block. After each pass, the 15-valued output array is shifted by one value position for proper alignment with its associated row of the data matrix. The DCT matrix is then processed by an inverse discrete cosine transform subprocess that generates decoded display data. A second lookup table can be used for 2×4×8 DCT processing.