A system for controlling the elevators in an elevator system based on passenger presence. The elevator system includes a number of elevators (1A . . . 1H) wherein first sensor members (2, 2A . . . 2H), second sensor members and additional sensor members are arranged in a waiting area of each elevator (1A . . . 1H) on each floor (F, F1, F2, F3 . . . Fn). The first, second and additional sensor members are fitted to give information about the presence and number of passengers waiting to go either up or down for an elevator at least in the waiting area in question. A controller is provided for controlling the elevators. The controller receives information from the first, second and additional sensor members about the presence and number of passengers waiting for an elevator and to control the movement of the elevators of the elevator system utilizing the information received from the first, second and additional sensor members.