A stereo matching algorithm based on spherical perspective projection model for fisheye images which have severe distortions and a large field of view with 180° is presented. Only estimating the focal length and the principal point of two fisheye images, all feature points on two fisheye images are first projected onto two spherical perspective images. Then rotation matrix and translation vector between two spherical perspective images are computed to find two corresponding spherical image patches, which are mapped into two fisheye images. And these estimated parameters are refined through minimizing the re-projected error of all feature points on the spherical perspective images. The corrected correspondence points between two fisheye images are determined through the refined rotation and translation. Experiments with real fisheye images show that this method has satisfactory matching results for the severely distorted fisheye images.