Disclosed is a novel pivot structure for a seat belt buckle wherein a base plate of a seat belt buckle is connected to an anchor stay by way of a pin and spring means interposed therebetween in such a manner that the buckle may pivot relative to the anchor stay about the pin along the major surface of the anchor stay and, additionally, away from the major surface. The spring means biases the buckle toward a neutral position in which the buckle aligns with the axial line of the anchor stay. Thus, the seat belt buckle is maintained in a relatively rigid state relative to the anchor stay so that a tongue plate may be easily aimed and inserted thereinto. However, once the seat belt is securely fastened by the engagement of the tongue plate and the seat belt buckle, the buckle may pivot in any direction relative to the anchor stay and therefore will readily bend to conform to the body of the person wearing the seat belt and improve the comfort in wearing the seat belt.