In order to meet the high pointing precision of photoelectric detecting system, a kinematic calibration algorithm using penalized least square method and bicubic spline interpolation is proposed to decrease the pointing error. According to the structure component of the photoelectric detecting system, the parametric model of pointing error is established by multi- body system theory. In consideration of the non- parametric component, semiparametric model is established, and the model is calibrated by penalized least square method to get the modified model. A prototype experimental platform is built, and two groups of data are measured, which are collected to calibrate the model and verify the modified results. The experimental results demonstrate that the mean values of azimuth and elevation pointing error are reduced from 92.1185″, 75.9358″ to 2.7100″, 2.7755″. The standard deviation values of azimuth and elevation pointing error are also reduced from 21.6522, 15.1744 to 10.8645, 10.7305. The experiment and results can make it clear that the algorithm is able to improve the pointing precision and stability of pointing effectively. Besides, the linear and nonlinear relationships are considered simultaneously.