A security management apparatus includes a receiver, a printer, and a notifier. The receiver acquires, at an entrance of a facility, information about a visitor to a visiting destination of the facility. The information includes a stay permitted time at the visiting destination and at least one of an attribute, relation information, or a visiting purpose. The printer generates a portable medium that includes the information and the stay permitted time and records an image allowing distinguishing of content of the information. The notifier is configured to receive the stay permitted time from one or more readers disposed at the facility and is configured to read the stay permitted time from the portable medium in a noncontact manner. The notifier, in response to the stay permitted time being elapsed, warns the visitor who passes a disposition position of the one or more readers.