Access control system and method to distinguish between tailgate and piggyback. The method includes receiving, from a camera positioned at an access point, one or more images of a first person at the access point and determining based on credential information whether the first person is authorized to enter through the access point. In response to determining that the first person is authorized to enter through the access point, the method includes determining, based on a consent-to-follow gesture of the first person detected in the one or more images, a number of guests of the first person requested to enter through the access point. The method also includes determining, based on the one or more images, a number of persons entering through the access point and generating an alert indicating unauthorized entry when the number of persons is greater than the number of guests.