An adjusting magnetism shelter of a cutter holder has a stationary collar, a sliding assembly and a cap. The stationary collar has a disk and multiple sliding recesses. A first hole is defined in the disk. The sliding recesses are defined in the disk. The sliding assembly has multiple sliding bars. Each sliding bar is mounted in a sliding recess and has a magnetism bar and a moving shaft. The magnetism bar has a reflection surface. The moving shaft protrudes from each sliding bar. The cap covers the stationary collar and has a top board. The top board has multiple arc grooves. Each arc groove has an inner end and an outer end. The inner end of the arc groove is near the first hole. The outer end of the arc groove is far away from the first hole and is inserted into one of the moving shafts.