A pointer positioning method of a handheld pointer device, which includes capturing a first image frame containing a reference point to compute a first pointing coordinate according to the image position of the reference point in the first image frame; generating a cursor parameter of a cursor according to the first pointing coordinate; when the handheld pointer device enters a pointer-lock mode, records the first pointing coordinate and positions the cursor at the first pointing coordinate on a display apparatus; when the handheld pointer device exits the pointer-lock mode, captures a second image frame to compute a second pointing coordinate according to the image position of the reference point in the second image frame to obtain a displacement vector between the first and the second pointing coordinates; generating the cursor parameter and controlling the movement of the cursor according to the displacement vector and the first pointing coordinate.