A device for changing styli or probes in a coordinate measuring machine has a magazine (330) with several stations each holding a stylus or probe (320). A holder (310) on the quill of the coordinate measuring machine has an opening (313A), so that kinematic support elements (325) in the holder (310) can be moved underneath cooperating kinematic support elements (324) on the stylus (320). The holder (310) is then lifted up to engage the kinematic support elements, and the stylus can then be carried horizontally out of the magazine (30). During this operation, the kinematic support elements are biased into engagement initially by a leaf spring (335), and subsequently by a spring-loaded arm (314) in the holder (310), which bears against the top of a plate (322) of the stylus (320). When the holder (310) is engaged in the magazine (330), an extension plate (334) causes the arm (314) to swing out of the way. This arrangement enables the holder (310 ) to pick up the stylus (320) purely by translational movements of the holder (310), without the need for any separate motor or electromagnet to couple the stylus to the holder.