In palette-based coding, a video coder may form a so-called “palette” as a table of colors representing the video data of a given block. The video coder may code index values for one or more pixels values of a current block of video data, where the index values indicate entries in the palette that represent the pixel values of the current block. A method includes determining a number of entries in a palette, and determining whether a block of video data includes any escape pixels not associated with any entry in the palette. The method includes responsive to determining that the number of entries is one, and that the block does not include any escape pixels, bypassing decoding index values for the pixel values of the block, and determining the pixel values of the block to be equal to the one entry in the palette.