Embodiments of apparatuses and methods for multiplicand shifting in a linear systolic array modular multiplier are disclosed. In one embodiment, an apparatus includes two processing elements of a linear systolic array. One processing element includes multiplication logic, multiplicand shift logic, an adder, modulus logic, and modulus shift logic. The multiplication logic is to multiply a word of the multiplicand and a bit of the multiplier to generate a product. The multiplicand shift logic is to shift the word of the multiplicand. The adder is to add the product to a first running sum to generate a second running sum. The modulus logic is to conditionally add a word of a modulus and the second running sum. The modulus shift logic is to shift the word of the modulus. The next processing element includes logic to multiply the shifted word of the multiplicand and the next bit of the multiplier.