In one embodiment, a method includes accessing first sensor data from a first sensor worn on a first portion of a user's body and accessing second sensor data from a second sensor worn on a second portion of the user's body. The method includes determining, based on both the first sensor data and the second sensor data, one or more first features related to the user's activity and determining, based on the first features, an initial classification of the user's activity. When the initial classification indicates a class that includes one or more subclasses that are more distinguishable by one of the sensors, then a specific subclassification may be determined based on sensor data from only that one sensor. Otherwise, the classification of the user's activity may be based on the one or more first features that use data from both sensors.