A method and apparatus are disclosed from the perspective of a UE. In one embodiment, the method includes maintaining a first timing advance for a cell, wherein the first timing advance is associated with a first identifier. The method further includes performing a random access procedure in the cell. The method also includes receiving an initial value of a second timing advance for the cell via a random access response of the random access procedure. In addition, the method includes receiving a second identifier to associate with the second timing advance. Furthermore, the method includes maintaining the first timing advance and the second timing advance for the cell.