The present subject matter is directed to a rotor blade assembly for a wind turbine having an improved shear web configuration. The rotor blade assembly includes an upper shell member having a spar cap configured on an internal surface thereof and a lower shell member having a spar cap configured on an internal surface thereof. The shear web extends between the spar caps along a longitudinal length of the blade. Further, the shear web includes at least one pultruded component defining a hollow cross-section.