Various aspects of the present disclosure generally relate to wireless communication and techniques to prevent frequent beam switching. In some aspects, a user equipment (UE) may determine a duration of time that the UE dwells on a beam. The UE may update a data structure, stored in memory of the UE, to add or modify a stored duration value associated with the beam based at least in part on determining the duration of time that the UE dwells on the beam. The UE may determine that a condition associated with the stored duration value is satisfied. The UE may modify a priority of the beam in connection with a beam selection procedure based at least in part on determining that the condition associated with the stored duration value is satisfied. Numerous other aspects are provided.