An encoder employs a trellis coded quantization (TCQ) unit and a compression unit. The TCQ uses a set of polynomials that have been selected to maximize granular gain. The TCQ unit operates on a block of samples from a source. The compression unit compresses bit planes of the TCQ output, using parity check matrices of corresponding LDPC codes, to obtain corresponding syndromes. The parity check matrices are selected so their compression performance approaches close to the limit for Slepian-Wolf coding. A decoder employs a decoding unit and an estimation unit. The decoding unit decodes the syndromes using side information to produce an estimate for the TCQ output. The side information is correlated with the source. The estimation unit estimates the block of source samples using the estimated TCQ output and the side information. Trellis coded vector quantization may be used as an alternative to TCQ.