A tool bit holder for releasably supporting a plurality of tool bits in a supported position wherein each tool bit has an elongated bit body extending from a working end to a tool end. The tool end of the bit being shaped to be received by a tool and the working end being shaped to engage an object such as a fastener. The bit further including an elongated side wall extending between the working and tool ends. The holder includes a frame having a plurality of spaced bit receiving nests and each of these nests has a bit opening shaped to receive one tool bit. The nests further including a flexible member movable relative to the tool bit opening and movable between an engaged position and a disengaged position such that the nest engages the elongated wall of the tool bit when the flexible member is in the engaged position and the associated bit is in the supported position thereby preventing relative movement between the nest and the associated bit. Further, the nest allows the tool bit to move relative to the opening when in the disengaged position.