A server-based car security service provides for image and position information to be provided to the server. The server uses the image information to make an authorization determination with respect to the current user. When the user is determined by the server to be an unauthorized user, the server alerts the owner or primary user of the car by sending the alert to a remote user device. The user device displays the image of the current user, and the owner has the opportunity to indicate whether the user is actually an unauthorized user or whether the determination at the server was a misrecognition error. When the user is unauthorized, the owner can request real-time monitoring of the car's position, and also the image inside the car, as well as an outside-of-the-car image.