An apparatus comprises a sub-section of a pulse power drilling assembly including a transformer encircling a center flow tube through which a drilling mud is to flow for drilling a borehole into a subsurface formation based on periodic pulses of electrical discharges from the pulse power drilling assembly. The transformer comprises at least one primary winding that encircles the center flow tube; a core that encircles the at least one primary winding, wherein the core comprises an electrically non-conductive material; and at least one secondary winding that encircles the core.