A downhole tool, such as a packer or liner hanger, has a mandrel on which slips are disposed for engaging a surrounding tubular downhole. When the tool is being run in or out of the hole, at least one magnetic component magnetically retains the slip adjacent the mandrel so the slip can be held away from the surrounding tubular. When the tool is set downhole, the initial magnetic retention can be broken, and the slips can be moved away from the mandrel to engage the surface of the surrounding tubular.