A probe cap for use with an ultrasound probe including a head portion and an acoustic surface is disclosed. In one embodiment, the probe cap includes a body that defines a cavity sized for releasably receiving the head portion of the probe therein. The probe cap body further defines a hole that is proximate the acoustic surface of the head portion. A compliant spacer component is disposed in the hole. The spacer component can include a hydrogel and provides an acoustic path between the acoustic surface and a tissue surface of a patient. The spacer component includes a skin contact surface that defines a concavity and is deformable against the tissue surface. Additional embodiments disclose various probe cap and accompanying needle guide designs for use in assisting a clinician with ultrasound probe use and needle insertion into a patient.