An assembly is provided for an engine. This engine assembly includes an engine structure and a fuel injector. The engine structure includes an injector aperture and a fuel supply passage. The injector aperture extends longitudinally through the engine structure along a centerline. The fuel supply passage extends laterally within the engine structure to the injector aperture. The fuel injector is mated with the injector aperture. The fuel injector includes a nozzle passage and a nozzle orifice. The nozzle passage extends longitudinally along the centerline within the fuel injector to the nozzle orifice. The nozzle passage is fluidly coupled with the fuel supply passage.