This paper presents vehicle detection and tracking algorithms based on real-time background (RTB) and phase-correlation (PC) in the video sequence of urban highway with fixed camera. Firstly moving objects are obtained by subtracting RTB from serial images. After classification, PC is used to determine corresponding regions of moving objects between consecutive frames. The problems of vehicles' merging and splitting, sudden stop, and restart are also considered. Experiments show that the method is practical and can realize real-time detection and tracking of vehicles on highway.