The present disclosure provides systems, apparatus, methods, and computer-readable media that support adapting image processing based on the user activity performed while recording an image. In a first aspect, a method of image processing may include determining a type of activity based on sensor data, such as motion sensors and/or image sensor data. The type of activity may be provided to an image processing algorithm, such as an image/video stabilization algorithm, which adapts based on the activity type. For example, a rolling shutter correction may have a strength factor that is adapted to lower values when the type of activity is a panning motion when faces are present in the frame. As another example, another image stabilization (IS) algorithm such as an electronic image stabilization (EIS) algorithm may have adaptive behavior adjusted based on the type of activity. Other aspects and features are also claimed and described.