Provided is a viewing angle control system and an image display device which are capable of controlling the viewing angle of a display screen in a longitudinal direction and improving the visibility in a specific lateral direction. The viewing angle control system includes a polarizer, and a light absorption anisotropic layer, in which the polarizer has an in-plane absorption axis in a film, an angle θ between a transmittance central axis of the light absorption anisotropic layer and a normal line of the film is in a range of 0.1° to 45°, and an angle φ between a direction in which the transmittance central axis of the light absorption anisotropic layer is orthographically projected onto the film surface and the absorption axis of the polarizer is 0° or greater and less than 85°, greater than 95° and less than 265°, or greater than 275° and 360° or less.