A method and apparatus of palette coding for video data having color formats including a monochrome format are disclosed. The system determines whether the video data is monochrome or non-monochrome. If the video data is monochrome, one or more single-component values corresponding to one or more palette coding parameters are parsed from or signaled in the video bitstream for a single color component of input monochrome video data. Palette decoding is then applied to the single color component of input monochrome video data using the palette coding parameters. If the video data is non-monochrome, one or more multi-component values corresponding to one or more palette coding parameters are parsed from or signaled in the video bitstream for multiple color components of input non-monochrome video data. Palette decoding is applied to the multiple color components of input non-monochrome video data using the palette coding parameters.