Disclosed herein are an apparatus and a method for recognizing a formalized character set based on weakly supervised localization. The formalized character set recognition apparatus based on weakly supervised localization may include memory for storing at least one program; and a processor for executing the program, wherein the program performs recognizing one or more numerals present in a formalized character set image and a number of appearances of each of the numerals, extracting a class activation map in which a location of attention in the formalized character set image is indicated when a specific numeral is recognized, and outputting a formalized character set number in which numerals recognized based on the extracted class activation map are arranged.