Methods and apparatus are provided for scheduling parallel jobs in a system of virtual containers. At least one parallel job is assigned to a plurality of containers competing for a total capacity of a larger container, wherein the at least one parallel job comprises a plurality of tasks. The assignment method comprises determining a current utilization and a potential free capacity for each of the plurality of competing containers; and assigning the tasks to one of the plurality of containers based on the potential free capacities and at least one predefined scheduling policy. The predefined scheduling policy may comprise, for example, one or more of load balancing, server consolidation, maximizing the current utilizations, minimizing a response time of the parallel job and satisfying quality of service requirements. The load balancing can be achieved, for example, by assigning a task to a container having a highest potential free capacity.