A card edge connector includes an elongated insulative housing, a number of electrical contacts received therein and an ejector rotatably mounted thereon. The insulative housing defines a central slot for receiving a card therein. At least one tower is formed at one end of the insulative housing and includes a pair of sidewalls. The ejector includes a main body with a kicker at a bottom end and a handle at a top end thereof. Each sidewall of the tower includes a first bar and a second bar engaging with a first protrusion disposed on the ejector thereby generating a two-segment interference between the ejector and the insulative housing.