Features are disclosed for accurately authenticating a delivery agent for unattended delivery of an item. The systems and methods described confirm the location of the delivery agent and proximity to the delivery location using short range wireless communications between a monitoring device at the delivery location and a communication device associated with the delivery agent. Access may also be conditioned on user specified “do not disturb” rules indicating when remote access is authorized. The authentication may be dynamically assessed such as based on a type of item being delivered.