The present invention relates to the measurement of vital signs such as a respiratory rate or a heart rate. In particular, a system (1) for determining a vital sign of a subject (100), comprising an imaging unit (2) for obtaining video data of the subject, a marker (10, 20, 60, 61) directly or indirectly attached to a body of the subject, wherein the marker comprises a graphical pattern (11, 21), an image processing unit (3) for detecting said marker in said video data, and an analysis unit (4) adapted to extract a vital sign parameter related to the vital sign of the subject from said video data and to determine the vital sign from said vital sign parameter. Further aspects of the invention relate to a device and a method for determining a vital sign of a subject and a computer program for carrying out said method.