The present invention is applicable to the field of videos. Provided are an anti-shake method and apparatus for a panoramic video, and a portable terminal. The method comprises: acquiring, in real time, a current state timestamp, an accelerometer numerical value and an angular velocity numerical value of a gyroscope in a portable terminal; performing estimation using extended Kalman filtering combined with the accelerometer numerical value and the angular velocity numerical value to obtain an amount of rotation of the portable terminal to a world coordinate system; synchronizing the timestamp of the gyroscope with a timestamp of a panoramic video frame; performing quaternion interpolation on the state of the gyroscope to acquire a rotation matrix corresponding to the panoramic video frame; and rotating a panoramic image according to the current rotation matrix to generate a stable video frame. By means of the present invention, a more accurate rotation matrix can be obtained, and a panoramic image is then rotated according to the current rotation matrix to generate a stable video frame. Therefore, a shaky video frame can finally be stabilized, VR motion sickness can be alleviated, and strong robustness for a loud-noise scene and most motion scenes is provided.