A method of and an apparatus for controlling intra and/or inter prediction for decoding of a video sequence are provided. The method includes determining a total number of intra prediction modes for a current block, a number of Most Probable Modes (MPMs) in the current block, and a number of non-MPMs in the current block, wherein the total number of intra prediction modes for the current block is equal to a sum of the number of MPMs in the current block and the number of non-MPMs in the current block. The method further includes adjusting the total number of intra prediction modes to a value such that the number of non-MPMs is a power of 2 or a sum of multiple power of 2.