A game system as an information processing system includes a game apparatus and a controller, and the controller is attached with an attachment device. In a case that a slit plate of the attachment device is removed, the controller detects positions of high-intensity parts as to infrared ray output from markers provided on a monitor, and designates coordinates on the monitor screen thereby. Alternatively, in a case that the slit plate is loaded, when a card printed and so forth with a dot pattern is slid along a guide groove of the attachment device, a position of a high-intensity part of a plurality of dots to which infrared ray is irradiated is detected, so that information processing based on the dot pattern is executed.