An example device for decoding video data includes a processor configured to decode first and second reference pictures, wherein the first reference picture and the second reference picture are either both to be displayed before or after the current picture, determine a first motion vector (MV0) from a current block of the current picture to a first reference block of the first reference picture, determine a second motion vector (MV1) from the current block to a second reference block of the second reference picture, determine a first temporal distance (TD0) from the current picture to the first reference picture, determine a second temporal distance (TD1) from the current picture to the second reference picture, and decode the current block according to bi-directional optical flow (BIO) using MV0, MV1, TD0, and TD1.