The software compensation techniques are investigated to improve the alignment accuracy in the alignment system for layered manufacturing process. The filtering algorithm with little degradation on image, which combines the SUSAN with multi-frame averaging, is introduced to alleviate the mark image distortion due to the filtering processing. The focus measure function based on Power Spectrum in Frequency Domain (PSFD) is adopted as the auto-focus function of the system and then the positioning error due to defocus is reduced or eliminated. Concerning the calibration residual error and its repeatability of various models, the cubic polynomial model is used to describe and calibrate the geometric distortion of mark image and the alignment error due to image geometric distortion is less than 0.25 μm after calibration. The alignment error due to mechanical instability of the alignment setup is reduced to less than 1.3 μm through mechanical swing trace fitting and error compensation. Finally, adopting the hybrid programming method with Delphi and Matlab, the software functions mentioned above are implemented and integrated.