A portable device and a method for controlling brightness thereof are provided. The portable device includes a display, a sensor and a controller. The display provides a display region that is displayed with a first brightness. The sensor senses a surrounding brightness of the portable device. The controller controls the display to display a partial region corresponding to a User Interaction (UI) element in the display region with a second brightness based on the sensed surrounding brightness.