A method and apparatus of Intra prediction mode coding using one or more extended neighbouring blocks are disclosed. The set of extended neighbouring blocks comprises at least one neighbouring block not corresponding to a top-left neighbouring block and not corresponding to a leftmost-above neighbouring block. One or more Intra prediction mode candidates derived based on Intra prediction modes of the set of extended neighbouring blocks related to the current block are used to predict the current Intra prediction mode of the current block. The set of extended neighbouring blocks may comprise spatial neighbouring blocks of the current block. The set of extended neighbouring blocks may also comprise temporal neighbouring blocks within a collocated block of the current block. Another method and apparatus for unified candidate Intra prediction mode list for different colour components are disclosed.