A method for a three-dimensional encoding or decoding system incorporating restricted sub-PU level prediction is disclosed. In one embodiment, the sub-PU level prediction associated with inter-view motion prediction or view synthesis prediction is restricted to the uni-prediction. In another embodiment, the sub-PU partition associated with inter-view motion prediction or view synthesis prediction is disabled if the sub-PU partition would result in sub-PU size smaller than the minimum PU split size or the PU belongs to a restricted partition group. The minimum PU split size may correspond to 8×8. The restricted partition group may correspond to one or more asymmetric motion partition (AMP) modes.