Techniques for performing motion recognition with minimum delay are disclosed. A processing unit is provided to receive motion signals from at least one motion sensing device, where the motion signal describes motions made by a user. The processing unit is configured to access a set of prototypes included in a motion recognizer to generate corresponding recognition signals from the motion signals in response to the motion recognizer without considering one or more of the prototypes completely in the motion recognizer. Movements of at least one of the objects in a virtual interactive environment is responsive to the recognition signals such that feedback from the motions to control the one of the objects is immediate and substantially correct no matter how much of the motion signals have been received.