To solve the problem of complicated initialization in existing parametric calibration models of the Scheimpflug camera, a new method based on general non-parametric imaging model is presented, which avoids the complicated initialization process and models the Scheimpflug camera as a set of image pixels and their associated projection rays in space. Thus, the calibration underlying general non-parametric imaging model simply refers to computation of the mapping between the pixels and the corresponding three-dimensional projection rays. Based on the assumption that multiple control points in respective local coordinate systems corresponding to same the pixel from different views should be collinear in a certain common coordinate system, a two-step calibration method using checkerboards is presented. A rough calibration is performed by using images of checkerboards with a sufficient overlap, and the rest image with largest overlap is iteratively added to calibration process to the complete the initial calibration. Then, a refinement of calibration is carried out through bundle adjustment. The real data experiments including calibration, reconstruction and pose estimation are performed, and the results demonstrate that the presented method is flexible, effective and accurate.