Embodiments of the present invention provide a motion prediction or compensation method during a video coding and decoding process. A motion prediction or compensation method provided in the embodiments of the present invention includes: acquiring a candidate motion vector set; conducting a refined search for an optimal motion vector based on information of the candidate motion vector set; and performing motion prediction or compensation by using the motion vector acquired by the refined search. This improves coding performance and meanwhile maintains reasonable complexity.