This apparatus has plural fixed grinding machines (10,11), which are installed along the direction of travelling line of billets (1). And, facing pair of grinders (3,4) are equipped in a manner that the distances between each facing pair of grinders (3,4) are gradually shorter along the travelling direction of the billets (1) from the upstream side to the downstream side. In such a multiple-stage grinding machine, the burr on the welded part of the top and bottom side of the billet (1) are ground sequentially by the grinders (3,4), and the burr on welded part of the right and left side of the billet are ground sequencially by the grinders (3,4).