A new method to correct the image distortion of a fisheye lens was presented. The relationship of radial position of the chief ray at a projection plane and image plane was obtained by tracing the chief ray through the optical system, then a Fourier series was applied to fit the relation curve. Finding the inverse function of the expression of the series, the image without distortion can be obtained in terms of the resultant distorted image. Correction of image distortion of two fisheye lenses was simulated numerically with the method, experiments of distortion correction of two pictures taken with one of fisheye lenses, Nikon 16 mm/F2.8 lenses, were performed. The results show that residual radial height error of the restored image to the object is quite small(less than 0.25%) when the projection plane coincides with the object plane. The experiments also demonstrate that the method is feasible.