The three-dimensional (3D) coordinate measuring system based on laser multi-lateration can realize the non-target self-calibration. Its measurement accuracy is influenced by the self-calibration accuracy. However, the layout of the system is one of main impact factors on the self-calibration accuracy. By theoretical analysis of the non-target self-calibration model, the result of the system layout optimization is the tri-rectangular pyramid layout. Simulation results show that the tri-rectangular pyramid layout can improve the self-calibration accuracy, thus can ensure the system accuracy.