The interaction between the specimen and the scanning probe in the near-field interferometry was modeled with a coupled point-dipole system, and a set of equations of different electric field components were used to calculate the intensity and phase images of the specimen. It was found that, the resolution of the near-field images was strongly dependent on both the polarization direction of the reference light and the position of the far-field detector and vice versa. For a specimen under given illumination, its near-field images changed remarkably with the slight adjustment in the detector position or the polarization of the reference light. In the case of evanescent illumination, the resolution of differently polarized images increased with the view angle of the detector to the specimen, and highly accurate images could be realized only when the detector was placed at a large enough view angle. Among the images of differently polarized components, only the image polarized normally to the specimen surface indicated the specimen in details with enough intensity.