A system and method can control exhaust braking in a vehicle. The vehicle includes an engine system. The engine system includes internal combustion engine, an intake manifold, a control module, an exhaust system, and a variable geometry turbocharger (VGT) having a turbine. The turbine includes turbine blades and vanes movable with respect to the turbine blades. The method includes the following: (a) receiving an exhaust brake torque request; (b) determining target pumping losses in the internal combustion engine based on the exhaust brake torque request; (c) determining a target exhaust gas pressure within the exhaust system based on the target pumping losses; and (d) determining a target vane position of the vanes based on the target exhaust gas pressure, wherein the target vane position yields an exhaust brake torque in accordance with the exhaust brake torque request.