A refrigerator appliance may include a cabinet, a door, a sealed cooling system, and an ice cream unit. The door may include a door liner defining an icebox compartment. The door may include a chilled air supply opening and a chilled air return opening spaced apart from the chilled air supply opening. The ice cream unit may be disposed within the icebox compartment. The ice cream unit may include a freezing tank, a rotatable auger extending into the freezing tank, a tank jacket, and a unit duct or support base. The tank jacket may receive the freezing tank and define a cooling passage extending about the freezing tank from a jacket inlet to a jacket outlet. The jacket outlet may be open to the icebox compartment to exhaust air from the cooling passage to the icebox compartment.