A pilot valve system for a subsurface safety valve operated by control fluid pressure from the surface including a pilot valve connected with the control fluid line to the subsurface safety valve and into the well production string immediately above the safety valve to bypass the control fluid pressure directly into the tubing string and dump the control fluid pressure from the subsurface safety valve into the tubing string directly above the valve to minimize the time delay between control fluid pressure reduction and the safety valve closure. Three embodiments of the pilot valve are disclosed. One embodiment is operable by electrical energy from the surface. The other embodiments are operable by acoustic energy and radio waves, respectively. Also disclosed is a minimum backlash latch assembly for releasably locking the pilot valve, or other well tools, along a well bore in a receptacle such as a side pocket mandrel.