During the calibration for a large field of view camera, it usually appears such a situation that the scene is too simplex to satisfy the condition of scene and motion constraints for self-calibration method, the strong three-dimensional condition for three-dimensional calibration and the condition of absolute coplane for flat target plate calibration. It is difficult to fulfill the mentioned conditions while calibrating a camera pointing to the upper-air area. So it calls for a more flexible algorithm when a large field of view camera is calibrated. A calibration method for large field of view camera based on infinite homography is pointed out which just needs approximate position of the camera and at least four noncollinear control points to solve the infinite homography. A coordinate transformation method is also proposed to ensure the stability during the linear solution and optimization process of the infinite homography. The initial camera parameters for Levenberg-Marquardt (LM) optimization algorithm are obtained by decomposing the infinite homography. During the optimization process, the hypothesis that image center is the principal point and the adoption of the first-order radial distortion model will lead to the augment of the redundant freedom, so that the parameters optimization can be realized by using just four image points. The condition needed by the proposed method is facile to satisfy compared with strong three-dimension or absolute coplane. The simulation and actual experiments not only prove the correctness and high precision of this method, but also testify that repeated measurement experiments can be easily and flexibly implemented by this method.