Pattern measurement accuracy is mainly depended on polarization analyzers array perpendicular error. Based on a developed four-channel time division multiplexing skylight polarization measuring system, a polarization perpendicular error model is proposed. Effects of error on the measurement accuracy are analyzed theoretically and investigated experimentally. The results show that when the polarization perpendicular error is reduced, the measurement accuracy of skylight polarization patterns and system positioning accuracy are improved greatly. When the error exists and cannot be eliminated, the adjustment of initial angles of polarization analyzers can improve dramatically the measurement accuracy of the system.