A method for antenna subset selection by joint processing in RF and baseband in a multi-antenna systems. Lt input data streams are generated in a transmitter for either diversity transmission or multiplexing transmission. These streams are modulated to RF signals. These signals are switched to the t branches associated with the t transmit antennas, and a phase-shift transformation is applied to the RF signals by a t×t matrix multiplication operator Φ1, whose output are t≧Lt RF signals. These signals are transmitted over a channel by t antennas. The transmitted signals are received by r antennas in a receiver. A phase-shift transformation is applied to the r RF signals by a r×r matrix multiplication operator Φ2. Lr branches of these phase shifted streams are demodulated and further processed in baseband to recover the input data streams.