A power tool with a quick release tool holder has a spindle with a central bore to receive a tool, a first aperture and second aperture extend into the central bore. A ball is in each of the first apertures. The ball locks the tool against axial movement. An elongated roller is in each of the second apertures. The roller prohibits rotation of the tool within the spindle. A first sleeve is fixed with the spindle. The first sleeve maintains the rollers in the apertures. A biasing member axially biases the rollers. A second sleeve is circumferentially positioned about the spindle and axially movable on the spindle. The second sleeve has a surface in contact with the balls. Also, a biasing member is provided to provide a force on the second sleeve such that the sleeve is moved against the force of the spring to enable insertion or removal of the tool.