An optical scrolling module includes a prism, a light source, an image sensor and a first condenser. The prism is disposed at an opening of an upper shell of a mouse and has a light incident surface, a reflective surface, a contact surface and a light emitting surface. The contact and light incident surfaces are respectively connected between the reflective and light emitting surfaces. The contact surface is exposed by the opening. A light beam from the light source is transmitted to the contact surface via the light incident and reflective surfaces. When an object with a pattern structure puts on the contact surface, the light beam is scattered and a portion of the light beam is transmitted to the image sensor via the first condenser. When the object is removed from the contact surface, the light beam is reflected to the light emitting surface by the contact surface.