A gradient computation unit computes a gradient of an objective function in a variable to be optimized. An added variable selection unit adds a variable corresponding to a largest absolute value of the computed gradient from among variables included in a variable set, to a nonzero variable set. A variable optimization unit optimizes a value of the variable to be optimized, for each variable included in the nonzero variable set. A deleted variable selection unit deletes a variable that, when deleted, causes a smallest increase of the objective function from among variables included in the nonzero variable set, from the nonzero variable set. An objective function evaluation unit computes a value of the objective function for the variable to be optimized.