An oil shale formation may be treated using an in situ thermal process. Heat may be applied to the formation to raise a temperature of a portion of the formation to a pyrolysis temperature. Heat input into the formation may be controlled to raise the temperature of portion at a selected rate during pyrolysis of hydrocarbons within the formation. A mixture of hydrocarbons, H2, and/or other formation fluids may be produced from the formation. The mixture may be separated into condensable hydrocarbons and non-condensable hydrocarbons. The condensable hydrocarbons removed from the formation may be a high quality oil that has a relatively low olefin content and a relatively high API gravity.