A structure and a formation method of a semiconductor device are provided. The method includes forming an adhesive layer over a semiconductor substrate and forming a magnetic element over the adhesive layer. The method also includes forming an isolation element extending across the magnetic element. The isolation element partially covers the top surface of the magnetic element and partially covers sidewall surfaces of the magnetic element. The method further includes partially removing the adhesive layer such that an edge of the adhesive layer is laterally disposed between an edge of the magnetic element and an edge of the isolation element. In addition, the method includes forming a conductive line over the isolation element.