A heterodyne interferometric ellipsometer (IE) with two acousto-optical modulators was investigated. A single layer of transparent ITO on a glass substrate was measured; the measurement errors of film thickness and refractive index range up to 4 nm and 6%, respectively. Beam splitters are also important error sources besides laser source and polarizing optical components. The influence of the depolarization effect and misorientation of beam splitters (BSs) on ellipsometry is studied. The error model is given by using Jones Vectors; the rules between errors and the optical characteristics and misorientation of BSs are calculated. The calculation results indicate the error of film thickness measurement produced by BSs is up to several nanometers, and is approximately linear to the misorientation angle. The influence of the depolarization effect and misorientation on measurement accuracy is cross-correlated, so the error can not be eliminated by a calibration process of removing the sample out of the optical system. For obtaining subnanometers accuracy, the misorientaion angle must be lower than 0.01°. The rules between the depolarization parameters and the nonlinear errors presented in this paper could be used to determine how to design or select beam splitters in a heterodyne interferometric ellipsometer.