Technology is described for migrating connections for Internet of Things (IoT) devices. Data packets may be received from an IoT device for forwarding to a first physical host in a computing service environment. A determination to migrate the connection for the IoT device from the first physical host to a second physical host in the computing service environment may be performed when a throughput of the data packets received from the IoT device for the first physical host exceeds a defined threshold. A connection migration procedure may be initiated to migrate the connection for the IoT device from the first physical host to the second physical host. The connection migration procedure may include an exchange of session state information for the IoT device from the first physical host to the second physical host to enable the migration of the connection to the second physical host.