The invention relates to a grinding device for grinding a concrete segment. The grinding device is arranged at the front end of a shield or a pipe jacking machine, and comprises a cutter disc and a driving shaft for driving the cutter disc, wherein a plurality of grinding cutters are arranged on the cutter disc, a grinding surface is formed on the surface of the cutter disc by the grinding cutters, and the grinding surface is a convex surface, so that when the grinding device grinds the concrete segment, the grinding cutters positioned at the center of the grinding surface make contact with the concrete segment earlier than the grinding cutters positioned at the edge of the grinding surface; and the driving shaft is connected between the cutter disc and the shield or the pipe jacking machine. The grinding surface of the cutter disc is designed to be the convex surface, and when the concrete is ground, the grinding cutters at the center of the grinding surface makes contact with the concrete segment earlier than the grinding cutters at the edge of the grinding surface to carry out grinding so as to prevent the edge of the pre-grinding part of the concrete segment from collapsing after being ground.