Provided here are various invert emulsion drilling fluid compositions. The invert emulsion fluid is a water in oil emulsion, which can include an invert emulsifier to stabilize the water in oil emulsion, a 16 to 18 carbon carboxylic acid, a 36 carbon fatty dimer diol, a polymeric filter control agent, and an inorganic mineral including one or more of lime, calcium chloride, and barite. The invert emulsion drilling fluids can be formulated to be substantially free of clay.