A method for world objects tracking and prediction by an autonomous vehicle includes receiving, from sensors of the AV, a first observation data; associating the first observation data with a first world object; determining hypotheses for the first world object; determining a respective hypothesis likelihood of each of the hypotheses indicating a likelihood that the first world object follows the intention; determining, for at least one hypothesis of the hypotheses, a respective state; and in response to a query, providing a hypothesis of the hypotheses based on the respective hypothesis likelihood of each of the hypotheses. A hypothesis corresponds to an intention of the first world object and the respective state includes predicted positions of the first world object.