A vibration insulator which can compensate for axial eccentricity occurring in a fuel injection valve and suppress vibrations of the valve during operation of a combustion engine and a support structure for the valve. The vibration insulator is interposed between a step height portion of the valve and a shoulder portion. The step height portion is increased in diameter in a tapered fashion and inserted into an insertion hole of a cylinder head. The shoulder portion is annularly extended in an inlet portion of the insertion hole opposed to the step height portion. The vibration insulator includes an annular tolerance ring on an inner circumferential inclined face thereof with recessed tapered faces opposed to the tapered face of the step height portion and which abuts the tapered face. The taper angles of the tolerance ring and of the step height portion are set so as to be different.