The invention relates to a method for processing video signals from a video sensor, in order to extract 3d shape information about objects represented in the video signals, the method comprising the following steps: providing a memory in which objects are stored in a 3d shape space, the shape space being an abstract feature space encoding the objects' 3d shape properties, and mapping a 2d video signal representation of an object in the shape space, the coordinates of the object in the shape space indicating the object's 3d shape.