Embodiments of the present disclosure disclose an optical signal processing method and an optical cross-connect apparatus. The method includes: receiving, by an input port i of the optical cross-connect apparatus, a first optical signal; performing, based on the first optical signal by a transmit-end signal processing module i, first phase modulation processing and first frequency mixing processing to obtain a THz signal that carries signal information of the first optical signal; transmitting, by a transmit-end antenna i, the THz signal; receiving, by a receive-end antenna j, the THz signal; performing, based on the THz signal by a receive-end signal processing module j, second phase modulation processing and second frequency mixing processing to obtain a second optical signal that carries the signal information; and outputting, by an output port j, the second optical signal.