The present invention provides a system and the accompanying method for controlling physical objects placed on an interactive board with voice commands. The system includes multiple physical objects, each embedded with an identifier and a wireless communication module, an interactive board that is configured to recognize the identifier and location information of a physical object placed on the interactive board, a processor operatively connected to the interactive board, a memory unit operatively connected to the processor and configured to store the correlation relationship between identifiers of physical objects and names of physical objects, a wireless communication module operatively connected to the processor, and a voice input module operatively connected to the processor and configured to receives a voice command from a user. Once multiple physical objects are placed on the interactive board, the processor is configured to generate a command instruction for each of the physical objects, based on the voice command from the user and the identifier and location information of the physical objects.