To solve the problem of video-based security surveillance, a real-time method for abandoned object detection is proposed. First, frame-to-frame difference is used to obtain the pixel-wise properties of motion, and two backgrounds are constructed to detect dual foregrounds. Then, the pixel-wise properties and both foregrounds are integrated to maintain the dual backgrounds. Since the false alarms and occlusions necessarily occur and degrade the detection performance in practical applications, an evidence image is aggregated to validate the abandoned object. Experiments on several different video sequences demonstrate that the proposed method is effective in detecting abandoned objects from cluttered scenes. In addition, this method runs around 54 frames per second for a sequence with an image resolution of 352×288, which can be applied to real-time surveillance tasks.