A protocol processing apparatus and method in a mobile communication system operates a transmitting end according to whether transmission errors occur in a wireless section in consideration of conditions of the wireless section. A TCP processing apparatus and method of a base transceiver subsystem (BTS) in a mobile communication system can operate a TCP sending end in a persist or congestion override mode by setting a window size of the TCP sending end as ‘0’ if an Ack is not arrived from a TPC receiving end for a round trip time of a wireless link. Preferred embodiments of the present invention can reduce or prevent TCP performance degradation caused by congestion avoidance and control by preventing the TCP sending end from performing congestion avoidance and control by packet loss in the wireless link section. Also, the BTS updates the round trip time (RTT) of the wireless link when the Ack is arrived from the TCP receiving end for the RTT of the wireless link. Preferred embodiments can retransmit the packets to the TCP receiving end if packets that belong to the same window of the arrived Ack exist in the cache, thus to swiftly perform retransmission.