The invention relates to a method and system for recognizing physical objects. In the method an object is gripped with a gripper, which is attached to a robot arm or mounted separately. Using an image sensor, a plurality of source images of an area comprising the object is captured while the object is moved with the robot arm. The camera is configured to move along the gripper, attached to the gripper or otherwise able to monitor the movement of the gripper. Moving image elements are extracted from the plurality of source images by computing a variance image from the source images and forming a filtering image from the variance image. A result image is obtained by using the filtering image as a bitmask. The result image is used for classifying the gripped object.