A real-time tracking of surgical tools relative to a pre-operative surgical plan and intra-operative images involves an image-based registration and tool tracking registration. The image-based registration implements an integration of a plurality of images of an anatomical region of a body including a pre-operative scan image (31) (e.g., a 3D CT/MRI image), an intra-operative fluoroscopic image (42) (e.g., an 2D X-ray image) and an intra- operative endoscopic image (23) (e.g., an 2D arthroscopic image). The tool tracking registration implements a representation within the pre-operative scan image (31) and/or the intra-operative fluoroscopic image (42) of a tracking of one or more surgical tools within the intra-operative endoscopic image (23).