Detection of merge scenarios by an autonomous vehicle (AV) is disclosed. A system includes a memory and a processor. The memory includes instructions executable by the processor to, in response to detecting a merge scenario, identify an interacting object pair including a merging object and a crossing object, where the AV is the merging object; generate a yield hypothesis and a no-yield hypothesis; compute a yield reference path corresponding to the yield hypothesis and a no-yield reference path corresponding to the no-yield hypothesis; determine a yield likelihood of the yield hypothesis and a no-yield likelihood of the no-yield hypothesis; and operate the AV to merge or to wait until the merge scenario is no longer detected based on the yield likelihood and the no-yield likelihood.