In recent years, slightly off-axis digital holography which combines the advantages of off-axis and on-axis has been vigorously developed. In order to further improve its real-time performance, a synchronous slightly off-axis system based on the field of view (FOV) multiplexing technique has been applied. However, the spatial position of the holograms collected by this technology is unknown, which causes a spatial mismatch problem. In order to ensure the accuracy of the subsequent holographic reconstruction, it is necessary to perform a spatial mismatch calibration. The existing calibration methods can be roughly divided into: intensity-based calibration methods and phase-based calibration methods. Intensity-based calibration methods are susceptible to environmental noise, and phase-based calibration methods only have pixel-level accuracy. At the same time, none of the existing methods take into account the longitudinal position error caused by the sensor tilt.The existing spatial mismatch calibration methods can only achieve pixel-level calibration accuracy due to the limitation of principle, and are easily disturbed by the environmental noise. In this paper, a spatial mismatch calibration method for a sub-pixel-level simultaneous slightly off-axis digital holographic microscope system is proposed. In the modeling, the method not only analyzes the lateral position error caused by the image segmentation, but also further considers the longitudinal position error caused by the sensor tilt, and summarizes the calibration process as a nonlinear multi-variable optimization problem. In this paper, the particle swarm optimization algorithm is used to solve this optimization problem because of its simple structure, high convergence efficiency, and strong global search ability. In the calibration process, a phase-only wavefront based on the phase aberration is established, and the root mean square error of the phase-only wavefront is used as the target function to remove the influence of noise on the calibration accuracy. Simulation results show that the proposed method has sub-pixel accuracy, and experiment demonstrates the effectiveness of the method in the practical systems.