The existing fisheye correction methods have only utilized single image and are lack of enough information to correct distortion errors due to some factors, such as camera shaking and environment disturbing. An automatic and accurate fisheye image correction algorithm is presented based on the matched feature points between two fisheye images, which can automatically correct the shooting errors and preserve the vertical invariance of corrected images by using the estimated pitch angles. Experiment shows that the proposed method can eliminate the ghost of image stitched caused by the distortion errors, and enhance the effect of the stitched images.