A method and apparatus of Inter prediction for video coding performed in a video encoder or a video decoder are disclosed. In one method, a GMC (global motion compensation) indication indicating whether global motion compensation is enabled is signalled at the video encoder side or parsed from the video bitstream at the video decoder side. The current processing unit is encoded or decoded using one or more coding modes including a GMC mode if the GMC indication is indicates GMC is enabled. According to another method, if a GMC mode is enabled for a region, a number representing the total number of multiple global motion models allowed is determined for encoding or decoding the processing units in the region using multiple global motion models. In yet another method the GMC and local motion compensation are applied selectively or in a combined fashion.