A method of manufacturing semiconductor mirror wafers includes a double side primary mirror polishing step, a back side low brightness polishing step and a front side final mirror polishing step, wherein a silica containing polishing agent is used together with a polyolefin type fine particle material for the back side low brightness polishing. The method is capable of low brightness polishing of the back side, sensor detection of the front and back sides, suppression of generation of fine dust or particles from back side, thereby to increase the yield of semiconductor devices, manufacturing mirror wafers with higher flatness level, and higher productivity due to simplification of processes.