The present invention relates to an isolated polarization beam splitter or combiner, for joining light from different inputs into one common port, and for dividing a beam of light into orthogonal polarizations. In both modes of operation, the splitter/combiner provides isolation preventing transmission of light in a reverse direction. As a splitter, a beam of light is separated through a birefringent material into sub-beams of orthogonal polarization components, and each sub-beam is passed through a non-reciprocal polarization rotator to rotate the polarization so that a reflected beam, or other counter-transmitted light cannot return on the same path through the birefringent material to the source. As a combiner, two separate beams of light are launched with known orthogonal polarizations into a first birefringent material, passed through a non-reciprocal polarization rotator and then combined as orthogonal polarizations into a single output port. Advantageously, by providing the splitting or combining function in a same isolating device, insertion losses are reduced and the device is smaller and more cost effective. As a further advantage, a polarization beam splitter/combiner is provided in which the optical path length for the two separated orthogonal polarizations traveling through a birefringent crystal is equal, while still permitting the economical use of birefringent crystals with an uncollimated beam of light.