A modular connector assembly includes a housing having a mating end and a mounting end opposite the mating end. The mating end is configured to be mated with a header connector and the mounting end is configured to be mounted to a circuit board. The housing has a plurality of individual chambers separated by chamber walls. Contact assemblies are received in corresponding chambers. Each contact assembly has a dielectric body holding a differential pair of receptacle signal contacts configured to be terminated to the circuit board and mated with corresponding header signal contacts of the header connector. Each contact assembly has a ground shield coupled to an exterior of the dielectric body providing electrical shielding for the differential pair of receptacle signal contacts from other pairs of receptacle signal contacts.