Large optomechanical instruments are difficult to process in short time. In order to verify the correctness of instrument design, the similarity model test was used to replace the prototype research. Based on the similarity theorem, the similarity relationship between the prototype and the scaled model of the collimator system in the Thirty Meter Telescope (TMT) wide field spectrometer was first derived. The finite element models of the prototype and the 1∶3 scaled model were then established, and the modal analysis and the surface analysis were conducted based on these two models. The similarity model test and the analysis for the scaled model object were finally performed. At the scale of 1∶3, the vibration mode similarities of these two models conform to the theoretical analysis results. The root mean square value of surface wavefront errors for the scaled model is 0.065λ(λ is wavelength) in the working environment. The model test results prove that the scaled model of the collimator system can satisfy the requirements of an optical system, which can predict the dynamic responses and surface deformation changes of prototype. The study can provide technical accumulation for the formal research of subsequent projects.