A bone anchoring device is provided that includes an anchoring element with a shank to be anchored in a bone or a vertebra, a connection element for connecting at least two anchoring elements, the connection element being made of an elastic material, a receiving part being connected to the shank for receiving the connection element and connecting the connection element to the shank, a seat for the connection element the seat being provided in the receiving part, and a locking element being engageable with the receiving part for fixation of the connection element in the seat. At least the portion of the connection element which is received in the recess has a substantially smooth-surface. The locking element has an engagement structure which contacts the surface of the connection element in such a way that a form-fit connection is provided.