A method and apparatus of handling a radio link failure in a wireless communication system is provided. Upon detecting a radio link failure in a Radio Resource Control (RRC) connected state, a user equipment starts a radio link failure timer of which timer value is selected among at least one timer value for at least one RB. The user equipment attempts a RRC connection re-establishment while the radio link failure timer is running, and transitions to a RRC idle state when the radio link failure timer is expired.