A method and apparatus of performing a handover procedure in a wireless communication system is provided. The target eNodeB (eNB) receives a specific establishment cause of a user equipment (UE). The specific establishment cause among a plurality of establishment causes comprises emergency, high priority access, mobile terminated access, mobile oriented signaling, mobile oriented data, and delay tolerant access. The target eNB transmits a handover command message to the UE after receiving the specific establishment cause.