Method and apparatus of video coding are disclosed. According to one method, the left reference boundary samples and the top reference boundary samples are checked jointly. According to another method, selected original left reference boundary samples and selected original top reference boundary samples at specific positions are used for predictor up-sampling. According to yet another method, the horizontal interpolation and the vertical interpolation are in a fixed order regardless of a shape of the current block, size of the current block or both.