A method and apparatus are disclosed from the perspective of a first UE (User Equipment) to perform sidelink logical channel (SL LCH) establishment. In one embodiment, the method includes the first UE storing a list of sidelink configurations, wherein each entry in the list contains one sidelink configuration and at least one PC5 QoS identifier (PQI) associated with the one sidelink configuration. The method further includes the first UE selecting an entry in the list according to a PQI of a PC5 QoS flow from a sidelink service. The method also includes the first UE establishing a SL LCH for the PC5 QoS flow according to a sidelink configuration of the entry and assigning an identity for the SL LCH. In addition, the method includes the first UE transmitting information to a second UE for the second UE to establish the SL LCH, wherein the information includes at least the identity of the SL LCH, an identity of the PC5 QoS flow, and Transmission-Reception (TX-RX) aligned parameters included in the sidelink configuration. Furthermore, the method includes the first UE transmitting sidelink packet(s) from the PC5 QoS flow on the SL LCH.