Metal impurities such as mercury, copper, lead, zinc, iron and cadmium that are contained in trace amounts in sulfur that have been recovered from zinc concentrates by direct pressure leaching or other hydrometallurgical processes are removed effectively by contacting the molten sulfur, as it is held at a temperature not lower than the melting point of sulfur, with activated alumina by passage through a packed layer of activated alumina or a filter coated with activated alumina, or by mixing with activated alumina under stirring or by passage through a fluidized layer of activated alumina. The method is capable of reducing the sum of metal impurities in the sulfur to 50 ppm or below, with contents of mercury being reduced to 1 ppm or less, to provide sulfur having a purity of at least 99.99%, which is advantageous for use as a raw material in the production of sulfuric acid and in other chemical processes.