A flexible collet on a subterranean tool has sacrificial soft components to protect seal bores through which the collets have to compress to get through. The sacrificial components can be replaced when the tool is removed to the surface. In one embodiment, threaded fasteners are used alone or with washers for height adjustment such that the heads of the fasteners which are softer than the seal bore material ride on the seal bore and take the wear. The tool can ultimately be used to latch into shifting sleeves to move such sleeves to open or close wall ports. Alternatively axial ridges with beveled profile ends can be used or rolling members such as wheels or balls can be used to keep sharp edges off the seal bore. EDM method can be used to create multiple fingers with an axial ridge profile and rounded end transitions.