In one embodiment, a method includes creating an active container and a standby container for a single supervisor of an operating system at a network device, instantiating instances for active Kernel Loadable Modules (KLMs) for servicing the active container, instantiating instances for standby KLMs for servicing the standby container, wherein one or more of the standby KLMs comprise upgraded versions of the active KLMs, and switching over from the active container to the standby container to perform an in-service upgrade of the KLMs for the operating system. An apparatus and logic are also disclosed herein.