The invention provides a method and a system for detecting taxies carrying passengers but not running the meter based on audio and video recognition. The method includes the following steps: S1, collecting the driving state of a vehicle; S2, if the vehicle has been ignited and changes from a static state to a running state and the meter does not run, collecting videos of the front passenger seat and the rear seats respectively, otherwise, returning to S1; S3, carrying out face recognition based on the collected video data; S4, if a face is recognized, determining that the vehicle carries a passenger but does not run the meter, and saving a picture and a collected photo for use; and S5, if no face is recognized, collecting and processing audio data, comparing the audio data with the voice of the driver, and if the difference between the audio data and the voice of the driver is too large, determining that the audio data is the voice of a passenger (namely, the vehicle carries a passenger but does not run the meter), and saving the audio data for use. By collecting audio and video data in vehicles, extracting the photos of the front passenger seats and the rear seats, carrying out face and shape recognition and processing the recording data, the detection of taxies carrying passengers but not running the meter is well realized.