An apparatus configured to code video information includes a memory unit and a processor in communication with the memory unit. The memory unit is configured to store video information associated with a base layer and an enhancement layer. The video information comprises at least one enhancement layer (EL) block and at least one co-located base layer (BL) block. The co-located BL block has motion information associated therewith. The processor is configured to, in response to determining that the size of the EL block is smaller than a threshold size, either (1) use less than all of the motion information associated with the co-located BL block to code the EL block, or (2) refrain from using any motion information associated with the co-located BL block to code the EL block. The processor may encode or decode the video information.