The present disclosure relates to a vibration damping system and a method for estimating a cutting force of a machine tool using the same, and according to the present disclosure, disclosed is technology including a housing; a stator rotatably positioned at an arbitrary angle in an internal space of the housing; a rotor positioned in a space inside the stator and rotating around an axis of rotation; a spindle rotating with the rotor; a first expander applying an attractive force that pulls the stator when the stator rotates; a second expander applying an attractive force in an opposite direction to the attractive force of the first expander; a first compressor applying a repulsive force that pushes the stator; and a second compressor; to suppress the generation of vibrations of the machine tool, thereby improving machining quality and machining accuracy of a structure.