An accessory may be provided with a button controller having a microphone and switches. The switches may include dome switch members and metal switch terminals mounted in a switch module housing structure. The switch module housing structure may include one or more recesses in which electrical components are mounted. A conductive backplate may cover the recesses and may be coupled to the switch module housing structure. A conductive film may be attached to the switch module housing structure over the dome switch members. The conductive film and the conductive backplate may form an electromagnetic interference shield around the electrical components in the recesses. The switch module housing structure may include a non-conductive plastic overmolded onto a metal frame or may be formed from a first shot of non-conductive plastic and a second shot of conductive plastic to form an electromagnetic interference shield for the electrical components.