A manipulation controller is provided for reorienting an object by a manipulator of a robotic system. The manipulation controller includes an interface controller configured to acquire measurement data from sensors arranged on the robotic system, at least one processor, and a memory configured to store a computer-implemented method. The instructions of the method include acquiring measurement data from vision sensors and force sensors arranged on the robotic system, determining an input-output relation for the object based on a nonlinear static model representing input-output relationships between contact forces and movements of the object on the workbench, representing interaction between the object and the manipulator using complementarity constraints to capture the contact state between the object and the manipulator, formulating a representation for frictional stability of the object based on the non-linear static model at the external contacts with the workbench; formulating a bilevel optimization problem so as to maximize the frictional stability over a position trajectory of the object being manipulated on the workbench, estimating uncertainty value in physical parameters to be compensated by performing the bilevel optimization problem, solving the bilevel optimization problem using the non-linear optimization solver and generating control data with respect to a sequence of the contact forces being applied to the object by using the manipulator.