External field-of-view (FOV) stitching is an effective way to achieve an airborne hyperspectral imaging system with both a large field-of-view and a wide spectral sampling range. However, due to the independent installation of each module, the boresight angles between the corresponding VNIR module and SWIR module will change after a long period of equipment operation, and the change of boresight angles will negatively affect the data fusion effect. The overlap of FOV makes the calibration method based on the epipolar geometry and homography constraints ineffective in solving the boresight angles between the corresponding VNIR/SWIR modules. In this paper, an algorithm based on the reprojection error is proposed for an airborne hyperspectral imaging system with external field-of-view stitching to achieve self-calibration of the boresight angles and focal length between the VNIR/SWIR backends. The algorithm has been applied to the Airborne Multi-Modality Imaging Spectrometer (AMMIS). Experimental results show that the average error of the method is less than 0.2 pixels, and it is also well adapted to tilt-placed modules.