A package includes a device die having a substrate. A molding compound contacts a sidewall of the substrate. A metal pad is over the substrate. A passivation layer has a portion covering an edge portion of the metal pad. A metal pillar is over and contacting the metal pad. A dielectric layer is over the passivation layer. A package material formed of a molding compound or a polymer is over the dielectric layer. The dielectric layer includes a bottom portion between the passivation layer and the package material, and a sidewall portion between a sidewall of the metal pillar and a sidewall of the package material. A polymer layer is over the package material, the molding compound, and the metal pillar. A post-passivation interconnect (PPI) extends into the polymer layer. A solder ball is over the PPI, and is electrically coupled to the metal pad through the PPI.