A device for voltage-noise rejection and fast start-up is provided. It comprises a low-pass filter connected to a voltage source, a voltage-controlled switch connected in parallel with the low-pass filter, and an auxiliary start-up element connected to a DC-only voltage output. By using a transistor operating in the triode region and a capacitor with suitable capacitance, it is suitable for integration to form a low-frequency low-pass pole to suppress the noise in the reference current. The auxiliary start-up element overcomes the large turn on time caused by the low frequency low-pass pole. As there is no static current during normal operation, the power consumption for the device is low.