An electromagnetic reciprocating compressor has a body (1, 2), a piston (10) reciprocating in a cylinder (12) in the body and a piston guide (14) in the form of an insert which is radially located relative to the axis of the cylinder (12) by fitting against a registration surface (25a), the registration surface is on a body part (1) which also provides the internal cylinder surface (12a) of the cylinder. To improve the alignment of the insert (14), the registration surface (25a) and the internal cylinder surface (12a) are formed to their respective final diameters in the same machining operation by reference to a common axis. The piston guide (14) is axially located related to the cylinder (12) by being resiliently clamped between the stator (23) and an end portion (27a) of the body (1, 2) opposed to the stator (23) with a resilient member (27) interposed between the body end portion (27a) and the piston guide (14).