The invention relates to a method of controlling a wind turbine, the wind turbine comprising wind turbine blades attached to a rotor hub and a control system for pitching the blades relative to the rotor hub. The method comprises determining a wind speed and providing a normal pitch mode of operation to control the output power of the wind turbine, where the pitch mode of operation comprises pitch reference values in dependence of the wind speed. The output power of the turbine is controlled according to the normal pitch mode of operation as a function of the wind speed if the wind speed is lower than a first upper level wind speed threshold, and according to a modified mode of operation if the wind speed exceeds the first upper level threshold wind speed, wherein the modified mode of operation comprises decreasing the output power according to a de-rating function which is a function of time. The invention further relates to a control system for controlling a wind turbine according to the above mentioned control method.