A handheld inkjet printer includes an inkjet print head and a tip. One or more sensors measure the position of points on a curved surface that are physically touched by the tip while the tip is moved relative to the surface. Based on these measurements, a computer generates or modifies a computer model that specifies at least (i) position of the curved surface, and (ii) a target region of the curved surface on which a pattern is to be printed. In addition, the one or more sensors measure position and orientation of nozzles in the print head, while the handset is moved relative to the surface. The computers also calculate, based on the computer model and these additional measurements, which of the nozzles to fire at a different times, such that the pattern is printed on the target region as the handset is moved relative to the surface.