A radially loading ignition system for a muzzle loading weapon generally comprises a body having threads on its outer surface and a well sunk into one end for loosely holding a shotgun shell primer; a first opening through the side of the well for passing the primer into the well radially; and a rotatable cover enclosing the well and having a second opening which is aligned with the first opening for loading the primer and which is rotated out of alignment with the first opening for closing the ignition system. The ignition system is suitable for use in muzzle loading weapons using either an in-line firing system or a conventional swing hammer. A spare primer holder is also disclosed which comprises a semi-circular ring of elastomeric material adapted to snap onto a rifle scope or barrel and having a series of holes for frictionally receiving spare primers.