Techniques for performing motion recognition using generalized motion recognition capability are disclosed. At least two types of sensors are used to generate sensor signals about a motion made by a user. The sensor signals are then transformed to device-independent motion signals that may be used to build generalized motion recognizers or used with one or more generalized motion recognizers responsive to the sensors. At least one of objects in a virtual interactive environment is responsive to a recognition signal determined from the motion signal and the generalized motion recognizers.